Imagine you're building a customer support tool that needs to find the most relevant FAQ entry for an incoming question. A user types "my package hasn't shown up yet," and your system needs to match it against an FAQ that says "Where is my order?" Those two phrases share almost no words in common — a traditional keyword search would fail completely. But a human reading both sentences instantly knows they mean the same thing. Embeddings are how you give machines that same intuition.
Embeddings are one of the foundational ideas behind modern AI search, recommendation systems, and retrieval-augmented generation. Once you understand them, a whole category of powerful applications opens up: semantic search, duplicate detection, document clustering, and more. By the end of this lesson, you'll not only understand what embeddings are and why they work — you'll have written working code to generate them using two different tools: OpenAI's API and the open-source Sentence Transformers library.
What you'll learn:
- What an embedding is, in plain terms, and why representing text as numbers is so powerful
- How embedding models are trained to capture meaning
- How to generate embeddings using the OpenAI API
- How to generate embeddings locally using Sentence Transformers
- How to measure similarity between embeddings using cosine similarity
- When to choose one approach over the other
Prerequisites
You should be comfortable writing Python and have a basic sense of what an API is. Familiarity with lists and loops is enough math background — you don't need linear algebra. If you haven't set up the OpenAI API in Python yet, check out Using the OpenAI API with Python first, as we'll use that setup here.
What Is an Embedding, Really?
Let's start from scratch. A computer can't natively work with the word "cat." It needs numbers. The simplest approach is to assign each word an ID: "cat" = 1, "dog" = 2, "table" = 3. But this is useless for capturing meaning — ID 1 and ID 2 are no more related than ID 1 and ID 1,000,000.
A smarter approach is a one-hot vector: represent "cat" as a list that's all zeros except for a single 1 at the index corresponding to "cat." If your vocabulary has 50,000 words, every word becomes a list of 50,000 numbers, almost entirely zeros. This is mathematically workable but still doesn't encode meaning — "cat" and "kitten" look completely unrelated.
An embedding solves this by representing a word, sentence, or even an entire document as a dense list of floating-point numbers — typically hundreds or thousands of them — where the position of each number in the list encodes something about meaning. These lists are called vectors, and the space they live in is called a vector space or embedding space.
The magic is this: embedding models are trained so that things with similar meanings end up with similar vectors. "cat" and "kitten" will have vectors that are close together. "cat" and "carburetor" will be far apart. "my package hasn't shown up" and "where is my order" will be very close — much closer than their word overlap would suggest.
Key insight: An embedding is a compressed, meaning-aware numerical representation of text. The distance between two embeddings reflects the semantic distance between the things they represent.
Think of it like a map. Cities that are geographically close are physically close on the map. Embedding spaces work the same way, but the "geography" is meaning. London and Paris are close; London and Tokyo are further; London and "photosynthesis" are on different continents entirely.
How Embedding Models Learn
You don't define these coordinates by hand. Embedding models learn them by training on enormous amounts of text. The training objective forces the model to place semantically similar text near each other in the vector space.
One classic training approach is called contrastive learning. During training, the model sees pairs of sentences: some that are semantically equivalent (positive pairs) and some that are unrelated (negative pairs). The model is rewarded for making positive-pair embeddings close together and negative-pair embeddings far apart. Over billions of examples, the model develops a rich internal geometry of meaning.
Modern embedding models are typically transformer-based — the same architecture behind LLMs. When you pass text through an embedding model, it processes every token (you can read more about how text gets split into tokens in Understanding Tokens: How LLMs Tokenize Text and Why It Affects Your Inputs, Outputs, and Costs), runs it through many layers of attention, and then pools the resulting representations into a single fixed-length vector.
The output vector has a fixed dimensionality — for example, OpenAI's text-embedding-3-small outputs 1,536-dimensional vectors. Every piece of text you embed with that model, regardless of how short or long it is, comes out as a list of exactly 1,536 numbers.
Note: The dimensionality doesn't tell you which "direction" means what. The model doesn't have a dimension that explicitly means "positive sentiment" or "about animals." The geometry emerges from training and is distributed across all dimensions at once.
Measuring Similarity: Cosine Distance
Once you have two embeddings, how do you measure how similar they are? The most common approach is cosine similarity.
Think of each embedding as an arrow pointing from the origin into high-dimensional space. Cosine similarity measures the angle between those two arrows. If the angle is 0 degrees (arrows pointing the same direction), similarity is 1.0 — they mean exactly the same thing. If the angle is 90 degrees (perpendicular), similarity is 0.0 — they're unrelated. If they point in opposite directions, similarity is -1.0.
The formula is straightforward:
cosine_similarity(A, B) = (A · B) / (|A| * |B|)
Where A · B is the dot product and |A|, |B| are the magnitudes of each vector. In Python with NumPy, this is a few lines of code, and we'll write it shortly.
Tip: You don't need to memorize the formula. The key intuition is: higher cosine similarity (closer to 1.0) means more semantically similar. In practice, you'll use a library function rather than computing it by hand.
Generating Embeddings with the OpenAI API
Let's write some real code. First, install the OpenAI library if you haven't already:
pip install openai numpy
Now let's generate embeddings for a few sentences and compare them:
import os
import numpy as np
from openai import OpenAI
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
def get_embedding(text: str, model: str = "text-embedding-3-small") -> list[float]:
"""Generate an embedding for a single piece of text."""
response = client.embeddings.create(
input=text,
model=model
)
return response.data[0].embedding
def cosine_similarity(vec_a: list[float], vec_b: list[float]) -> float:
"""Compute cosine similarity between two vectors."""
a = np.array(vec_a)
b = np.array(vec_b)
return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))
# Test sentences
sentences = [
"My package hasn't arrived yet.",
"Where is my order?",
"I want to return a damaged item.",
"What is the capital of France?",
]
# Generate embeddings for all sentences
embeddings = {sentence: get_embedding(sentence) for sentence in sentences}
# Compare them pairwise
print("Similarity scores:\n")
for i, sent_a in enumerate(sentences):
for j, sent_b in enumerate(sentences):
if j <= i:
continue
score = cosine_similarity(embeddings[sent_a], embeddings[sent_b])
print(f" [{score:.3f}] '{sent_a[:40]}' vs '{sent_b[:40]}'")
When you run this, you'll see output roughly like:
Similarity scores:
[0.921] 'My package hasn't arrived yet.' vs 'Where is my order?'
[0.541] 'My package hasn't arrived yet.' vs 'I want to return a damaged item.'
[0.198] 'My package hasn't arrived yet.' vs 'What is the capital of France?'
[0.573] 'Where is my order?' vs 'I want to return a damaged item.'
[0.201] 'Where is my order?' vs 'What is the capital of France?'
[0.183] 'I want to return a damaged item.' vs 'What is the capital of France?'
Notice how "My package hasn't arrived yet" and "Where is my order?" score extremely high despite sharing zero meaningful words. And the geography question sits far away from all the customer service sentences. That's semantic understanding working exactly as intended.
Embedding Multiple Texts in One Call
The OpenAI API accepts a list of strings in a single request, which is much more efficient than calling it once per sentence:
def get_embeddings_batch(texts: list[str], model: str = "text-embedding-3-small") -> list[list[float]]:
"""Generate embeddings for a list of texts in a single API call."""
response = client.embeddings.create(
input=texts,
model=model
)
# The API returns results in the same order as input
return [item.embedding for item in sorted(response.data, key=lambda x: x.index)]
faq_questions = [
"Where is my order?",
"How do I return an item?",
"What payment methods do you accept?",
"How do I contact customer support?",
"Can I change my shipping address after ordering?",
]
faq_embeddings = get_embeddings_batch(faq_questions)
print(f"Generated {len(faq_embeddings)} embeddings, each of dimension {len(faq_embeddings[0])}")
# Output: Generated 5 embeddings, each of dimension 1536
Warning: Be mindful of rate limits and cost. The OpenAI embeddings API is priced per token. For large document collections, batch your requests and cache the resulting vectors — you should never re-embed the same text twice. You can learn more about managing these costs in Cost Optimization: Token Counting, Caching, and Model Selection for LLMs.
Which OpenAI Embedding Model Should You Use?
OpenAI currently offers two main embedding models:
text-embedding-3-small: 1,536 dimensions. Fast and cheap. Great for most use cases.text-embedding-3-large: 3,072 dimensions. Higher quality, particularly for nuanced retrieval tasks, but about 6x more expensive.
For most applications, start with text-embedding-3-small. The quality difference only becomes meaningful at scale or for highly technical domains.
OpenAI's API is convenient, but it requires an internet connection and incurs per-token costs. For many use cases — especially when you're processing large datasets or have privacy requirements — you want to run an embedding model locally. The Sentence Transformers library makes this straightforward.
pip install sentence-transformers
The library automatically downloads and caches model weights from Hugging Face. Here's the equivalent of our earlier example:
from sentence_transformers import SentenceTransformer
import numpy as np
# Load a model - this downloads it the first time (~90MB for this model)
model = SentenceTransformer("all-MiniLM-L6-v2")
sentences = [
"My package hasn't arrived yet.",
"Where is my order?",
"I want to return a damaged item.",
"What is the capital of France?",
]
# encode() returns a numpy array directly
embeddings = model.encode(sentences)
print(f"Shape: {embeddings.shape}")
# Output: Shape: (4, 384) <- 4 sentences, 384 dimensions each
Notice two things. First, Sentence Transformers returns a NumPy array directly rather than a list — so .shape works immediately. Second, all-MiniLM-L6-v2 produces 384-dimensional vectors, much smaller than OpenAI's 1,536. Smaller vectors are faster to compare and store, though potentially at some quality cost.
Let's build the same semantic search comparison:
from sentence_transformers import SentenceTransformer, util
model = SentenceTransformer("all-MiniLM-L6-v2")
faq_questions = [
"Where is my order?",
"How do I return an item?",
"What payment methods do you accept?",
"How do I contact customer support?",
"Can I change my shipping address after ordering?",
]
user_query = "I haven't received my package and it's been two weeks"
# Embed everything
faq_embeddings = model.encode(faq_questions, convert_to_tensor=True)
query_embedding = model.encode(user_query, convert_to_tensor=True)
# util.cos_sim handles batched cosine similarity
scores = util.cos_sim(query_embedding, faq_embeddings)[0]
# Rank results
ranked = sorted(zip(scores.tolist(), faq_questions), reverse=True)
print(f"Query: '{user_query}'\n")
print("Most relevant FAQs:")
for score, question in ranked:
print(f" [{score:.3f}] {question}")
Output:
Query: 'I haven't received my package and it's been two weeks'
Most relevant FAQs:
[0.734] Where is my order?
[0.421] Can I change my shipping address after ordering?
[0.387] How do I return an item?
[0.312] How do I contact customer support?
[0.201] What payment methods do you accept?
This is the core of a semantic search engine — and you ran it entirely on your local machine, for free, with no network calls after the initial model download.
The Hugging Face Hub has hundreds of sentence transformer models. Here are three solid starting points:
| Model |
Dimensions |
Speed |
Quality |
all-MiniLM-L6-v2 |
384 |
Very fast |
Good general purpose |
all-mpnet-base-v2 |
768 |
Moderate |
Better quality |
BAAI/bge-large-en-v1.5 |
1024 |
Slower |
Excellent, state-of-the-art |
Tip: For production systems where you're embedding millions of documents, all-MiniLM-L6-v2 is often the right choice — its speed advantage compounds dramatically at scale. For smaller collections where retrieval quality matters most, BAAI/bge-large-en-v1.5 consistently performs well on benchmarks.
| Consideration |
OpenAI API |
Sentence Transformers |
| Setup complexity |
Low (just an API key) |
Low (just pip install) |
| Cost |
Per token |
Free after download |
| Privacy |
Text leaves your infrastructure |
Fully local |
| Embedding quality |
Excellent |
Good to excellent (model-dependent) |
| Speed at scale |
Rate-limited |
Hardware-limited |
| Offline use |
No |
Yes |
Use the OpenAI API when you're prototyping quickly, you have modest volume, or you want the best possible out-of-the-box quality without tuning. Use Sentence Transformers when you're processing large datasets, working with sensitive data, need offline capability, or want to keep costs at zero. This decision framework is similar to the broader model selection choices discussed in OpenAI vs Anthropic vs Open Source: Choosing the Right LLM.
What Comes Next: From Embeddings to Real Applications
Generating a single embedding or comparing two sentences is just the beginning. The real power emerges when you store embeddings for thousands or millions of documents and search across them efficiently — that's the heart of Retrieval-Augmented Generation (RAG), where you retrieve relevant context before sending it to an LLM.
To go deeper on that, Retrieval-Augmented Generation Explained: How RAG Works and When to Use It is the natural next step. And if you want to see embeddings powering a complete end-to-end system, Building a Document Q&A System with Embeddings: A Complete Beginner's Guide walks you through the full pipeline.
There are also important practical details to work through: how you split long documents before embedding them (see Chunking Strategies for RAG: How to Split Documents by Size, Sentence, and Semantic Meaning) and how to store and query vectors efficiently at scale.
Hands-On Exercise
Build a minimal semantic FAQ matcher using either the OpenAI API or Sentence Transformers. Your program should:
- Define a list of at least 8 FAQ questions covering a realistic domain (e-commerce, HR policies, a software product — your choice)
- Accept a user query as input (either hardcoded or via
input())
- Embed both the FAQ list and the query
- Return the top 3 most relevant FAQs with their similarity scores
- Print a clear warning if the top score is below 0.3 (indicating the query probably doesn't match any FAQ)
Stretch goal: Store your FAQ embeddings in a dictionary and pickle it to disk so you don't re-embed the same questions on every run. Load from disk if the file exists, regenerate if it doesn't.
Common Mistakes & Troubleshooting
Mistake 1: Re-embedding the same documents on every run
This is the most common beginner error. Embeddings are deterministic — the same text always produces the same vector. Compute embeddings once, save them (to a .pkl file, a database, or a vector store), and load them on subsequent runs.
Mistake 2: Comparing embeddings from different models
Embeddings from text-embedding-3-small are not comparable to embeddings from all-MiniLM-L6-v2. They live in completely different vector spaces. If you switch models, you must re-embed everything.
Mistake 3: Embedding very long documents as a single unit
Embedding models have a maximum input length (typically 512 tokens for most Sentence Transformer models, up to 8,191 tokens for text-embedding-3-small). Text beyond this limit gets truncated silently. For long documents, split them into chunks before embedding. The chunking strategies article linked above covers this in depth.
Warning: Sentence Transformers will silently truncate text that exceeds the model's maximum sequence length without raising an error. Always check your document lengths relative to the model's max_seq_length attribute (model.max_seq_length) when working with longer texts.
Mistake 4: Using cosine similarity when you need exact deduplication
Cosine similarity is a measure of direction, not magnitude. Two texts can have a similarity of 0.97 and still be different documents. For exact duplicate detection, use a hash. Use embeddings for semantic near-duplicate detection where some paraphrase variation is expected.
Mistake 5: Forgetting to normalize scores for display
Raw cosine similarity scores can be unintuitive to end users. A score of 0.65 might be a great match for some models and a mediocre one for others. Always calibrate your thresholds empirically by checking scores against examples you understand, rather than assuming a universal cutoff.
Summary & Next Steps
Here's what you now understand:
- Embeddings are dense numerical vectors where semantic similarity corresponds to geometric closeness
- Embedding models are trained using contrastive objectives that push similar text together and dissimilar text apart
- Cosine similarity measures the angle between two embedding vectors and is the standard way to compare them
- The OpenAI API (
text-embedding-3-small) gives you excellent quality with minimal setup, at a per-token cost
- Sentence Transformers gives you free, local, private embedding generation with good-to-excellent quality depending on the model you choose
- Embeddings are the engine beneath semantic search, RAG pipelines, duplicate detection, and document clustering
From here, your most valuable next step depends on what you want to build. If you want to put embeddings to work in a real application, Building a Document Q&A System with Embeddings: A Complete Beginner's Guide is a natural continuation. If you're thinking about production scale, you'll need to understand how to handle long documents efficiently — Chunking Strategies for RAG: How to Split Documents by Size, Sentence, and Semantic Meaning covers exactly that.
Embeddings aren't an advanced topic to save for later — they're a foundation skill that unlocks a huge category of AI application patterns. The sooner they're in your toolkit, the more you'll find yourself reaching for them.
