Embeddings and vector stores
An embedding is a list of numbers that represents the meaning of a text, and a vector store is a database that returns the stored embeddings closest to the embedding of a question.
Last updated: 08 Oct, 2026 · LangChain 1.4
The chunks from Loading and chunking documents are still text. A search needs a way to say that two texts mean the same thing even when they share no words. Embeddings give that, and a vector store keeps them searchable.
Turning text into vectors
The video embeds with a local model, all-MiniLM-L6-v2, which gives 384 numbers per text. The code here calls Google's hosted gemini-embedding-001 instead, so no model is downloaded; it needs the GOOGLE_API_KEY from the setup lesson. Two methods do the work: embed_query for one text and embed_documents for a list.
Creating the embedding model
from langchain_google_genai import GoogleGenerativeAIEmbeddings
embeddings = GoogleGenerativeAIEmbeddings(model="gemini-embedding-001")
vector = embeddings.embed_query("How many days of leave do I get?") # a list of floats
vectors = embeddings.embed_documents(["first text", "second text"]) # one list per textComparing vectors with cosine similarity
Cosine similarity measures the angle between two vectors: 1 means they point the same way, values near 0 mean unrelated. After scaling each vector to length 1, a matrix product gives every pair at once.
import numpy as np
unit = vectors / np.linalg.norm(vectors, axis=1, keepdims=True) # each row now has length 1
print(unit @ unit.T) # cosine similarity of every pairimport numpy as np
from langchain_google_genai import GoogleGenerativeAIEmbeddings
embeddings = GoogleGenerativeAIEmbeddings(model="gemini-embedding-001")
sentences = [
"How many days of paid leave do employees get?",
"Staff receive 24 days of annual vacation.",
"Attention maps a query and key-value pairs to an output.",
]
vectors = np.array(embeddings.embed_documents(sentences))
print(vectors.shape)
unit = vectors / np.linalg.norm(vectors, axis=1, keepdims=True)
print(np.round(unit @ unit.T, 3))(3, 3072) [[1. 0.891 0.707] [0.891 1. 0.687] [0.707 0.687 1. ]]
Reading the similarity matrix
- The shape (3, 3072) says three texts, each a vector of 3072 numbers, the size this model returns by default.
- The first two sentences score 0.891. One is a question about paid leave and the other an answer about vacation days, with few words in common, and they are the closest pair in the matrix.
- The attention sentence scores 0.707 and 0.687 against them, lower, as it is about a different subject.
- Unrelated text still scores high. Even the unrelated pairs are near 0.7, so compare scores with each other, not with 0 or 1.
Storing chunks in a vector store
The video wraps a Chroma database in a class: a client with a persistent folder, a collection to hold the records, and an add method that needs four things for each chunk, an id, the text, its metadata and its embedding. The code below keeps the same four things but uses LangChain's vector store interface, so the same few calls work on any store. InMemoryVectorStore keeps everything in Python memory and ships with langchain-core.
Building the store from chunks
from langchain_core.vectorstores import InMemoryVectorStore
store = InMemoryVectorStore.from_documents(chunks, embeddings) # embeds every chunk, keeps the vectorsSearching with a score
results = store.similarity_search_with_score("What is scaled dot-product attention?", k=3)
for doc, score in results: # the k closest chunks, best first
print(round(score, 3), doc.metadata["source"], doc.metadata["page"])This example uses two of the papers and four of their pages, so that the whole index stays a few thousand tokens. The later RAG lessons start from the same chunks.
import os, urllib.request
from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_google_genai import GoogleGenerativeAIEmbeddings
from langchain_core.vectorstores import InMemoryVectorStore
PAPERS = {"attention": "1706.03762", "qzhou": "2508.21632"}
PICK = {"attention": [3, 4], "qzhou": [0, 1]}
pages = []
for name, arxiv_id in PAPERS.items():
if not os.path.exists(f"{name}.pdf"):
urllib.request.urlretrieve(f"https://arxiv.org/pdf/{arxiv_id}", f"{name}.pdf")
docs = PyPDFLoader(f"{name}.pdf").load()
pages += [docs[i] for i in PICK[name]]
chunks = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200).split_documents(pages)
for i, c in enumerate(chunks):
c.id = f"c{i}"
print(len(pages), "pages ->", len(chunks), "chunks")
store = InMemoryVectorStore.from_documents(chunks, GoogleGenerativeAIEmbeddings(model="gemini-embedding-001"))
for q in ["What is scaled dot-product attention?", "Which model is QZhou-Embedding built on?"]:
print(q)
for doc, score in store.similarity_search_with_score(q, k=3):
print(" ", round(score, 3), doc.id, doc.metadata["source"], doc.metadata["page"], repr(doc.page_content[:40]))4 pages -> 15 chunks What is scaled dot-product attention? 0.788 c0 attention.pdf 3 'Scaled Dot-Product Attention\n Multi-Head' 0.779 c1 attention.pdf 3 'into a matrixQ. The keys and values are ' 0.77 c2 attention.pdf 3 'dk, the dot products grow large in magni' Which model is QZhou-Embedding built on? 0.797 c7 qzhou.pdf 0 'QZhou-Embedding Technical Report\n Kingso' 0.788 c9 qzhou.pdf 0 'Our model weights are released on Huggin' 0.785 c13 qzhou.pdf 1 'dient norms during optimization. Thus th'
Reading the search results
- Four pages became 15 chunks, with the ids c0 to c14 from the loop.
- Each result row is the score, the chunk id, the file, the page index and the start of the text. The scores are cosine similarities, so the best chunk has the highest number and comes first.
- The attention question returned c0, c1 and c2, all from page 3 of
attention.pdf, with scores 0.788, 0.779 and 0.77. Neighbouring chunks share text through the overlap, so three results can say nearly the same thing. - The QZhou question returned only chunks of
qzhou.pdf. The top one, c7 with 0.797, is the abstract, which says the model is built on Qwen2.5-7B-Instruct.
Stable ids and saving to disk with FAISS
A store needs an id for every record, and the id must be the same every time the same chunk is added; with random ids, running the indexing code twice adds a second copy of every chunk. The loop c.id = f"c{i}" in the example above gives each chunk a stable id, and a store that is given the same id twice can then refuse or replace instead of piling up copies.
FAISS is a similarity-search library from Meta that runs in your process and saves to a folder. LangChain wraps it as FAISS in langchain-community (pip install faiss-cpu). It stores vectors in a flat index and measures squared L2 distance (the straight-line gap between two vectors, squared), so a smaller number means closer; that is the opposite direction from a cosine score.
Building, saving and loading a FAISS index
from langchain_community.vectorstores import FAISS
store = FAISS.from_documents(chunks, embeddings, ids=[c.id for c in chunks])
store.save_local("faiss_demo") # writes index.faiss and index.pkl
again = FAISS.load_local("faiss_demo", embeddings, allow_dangerous_deserialization=True)from langchain_community.vectorstores import FAISS
from langchain_google_genai import GoogleGenerativeAIEmbeddings
embeddings = GoogleGenerativeAIEmbeddings(model="gemini-embedding-001")
store = FAISS.from_documents(chunks, embeddings, ids=[c.id for c in chunks])
print(store.index.ntotal, "vectors of", store.index.d, "numbers,", type(store.index).__name__)
for doc, distance in store.similarity_search_with_score("What is scaled dot-product attention?", k=3):
print(round(float(distance), 3), doc.id, doc.metadata["source"], doc.metadata["page"])
store.save_local("faiss_demo")
again = FAISS.load_local("faiss_demo", embeddings, allow_dangerous_deserialization=True)
print("loaded:", again.index.ntotal, "vectors")
try:
again.add_documents(chunks[:3], ids=[c.id for c in chunks[:3]])
except ValueError as err:
print("ValueError:", str(err)[:60])15 vectors of 3072 numbers, IndexFlatL2
0.423 c0 attention.pdf 3
0.442 c1 attention.pdf 3
0.46 c2 attention.pdf 3
loaded: 15 vectors
ValueError: Tried to add ids that already exist: {'c0', 'c2', 'c1'}Reading the FAISS run
- FAISS holds 15 vectors of 3072 numbers in an
IndexFlatL2, a flat index that compares the question with every vector. - The top three are c0, c1 and c2 again, as with cosine, now as distances 0.423, 0.442 and 0.46. The smallest number is the best match. The vectors have length 1, and for such vectors the squared distance equals 2 minus 2 times the cosine similarity: 2 − 2 × 0.788 = 0.424, 2 − 2 × 0.779 = 0.442 and 2 − 2 × 0.77 = 0.46, the three distances above up to rounding.
- Saving and loading round-trips. The loaded store has 15 vectors again.
save_localwrites two files,index.faissfor the vectors andindex.pklfor the texts and metadata. - Re-adding the same ids is refused. FAISS raises a
ValueErrorthat names c0, c1 and c2, so a re-run that repeats ids shows up as an error and not as silent copies.
Cosine similarity vs L2 distance vs dot product
| Cosine similarity | Squared L2 distance | Dot product | |
|---|---|---|---|
| Measures | The angle between two vectors | The squared straight-line gap between two vectors | Angle and length together |
| Closer means | A higher number | A lower number | A higher number |
| Where it shows up here | InMemoryVectorStore scores | FAISS default scores | Equals cosine when vectors have length 1 |
Where you use embeddings and vector stores
- Semantic search. Finding the chunk about "vacation" for a question about "paid leave", with no shared word.
- RAG retrieval. The vector store is the box in the middle of the pipeline that every question searches.
- Deduplication and clustering. Texts whose vectors are very close are near-copies, which helps to clean a corpus before it is indexed.
InMemoryVectorStore returns cosine similarity (higher is closer), FAISS returns squared L2 distance (lower is closer), and Chroma's default distance is squared L2, so 1 - distance is a cosine similarity only when the collection is created with the cosine space. Embed questions with the same model that embedded the chunks; vectors from two different models cannot be compared.Related
- Previous: Loading and chunking documents
- Next: Building a RAG pipeline
- Go deeper: Embeddings and vector stores
- Add a fourth sentence to the first example that mentions leave in other words, such as "Time off per year is 24 days", and compare its row of the matrix with the first sentence.
- Change
k=3tok=6in the search and read where the scores start to drop. - Search for a question that is not in the papers, such as "Who won the 2022 World Cup?", and look at the top score next to the one for a question the papers answer.
Slow is fine. Stopping is the only problem.