Generative AI and Agentic AILangChain 1.4 · LangGraph 1.2 · Python 3.12 or 3.13
Dashboard
0%

Embeddings and vector stores

An embedding is a list of numbers that represents the meaning of a text, and a vector store is a database that returns the stored embeddings closest to the embedding of a question.

Last updated: 08 Oct, 2026 · LangChain 1.4

The chunks from Loading and chunking documents are still text. A search needs a way to say that two texts mean the same thing even when they share no words. Embeddings give that, and a vector store keeps them searchable.

A two-dimensional sketch of an embedding space. Two sentences about vacation days sit close together, sentences about attention and about chunk size sit elsewhere, and a question about paid leave lands next to the two vacation sentences, which are the closest points a search would return.

Turning text into vectors

The video embeds with a local model, all-MiniLM-L6-v2, which gives 384 numbers per text. The code here calls Google's hosted gemini-embedding-001 instead, so no model is downloaded; it needs the GOOGLE_API_KEY from the setup lesson. Two methods do the work: embed_query for one text and embed_documents for a list.

Creating the embedding model

python
from langchain_google_genai import GoogleGenerativeAIEmbeddings

embeddings = GoogleGenerativeAIEmbeddings(model="gemini-embedding-001")
vector = embeddings.embed_query("How many days of leave do I get?")   # a list of floats
vectors = embeddings.embed_documents(["first text", "second text"])   # one list per text

Comparing vectors with cosine similarity

Cosine similarity measures the angle between two vectors: 1 means they point the same way, values near 0 mean unrelated. After scaling each vector to length 1, a matrix product gives every pair at once.

python
import numpy as np

unit = vectors / np.linalg.norm(vectors, axis=1, keepdims=True)   # each row now has length 1
print(unit @ unit.T)                                              # cosine similarity of every pair
ExampleAPI keyRun on gemini-embedding-001
import numpy as np
from langchain_google_genai import GoogleGenerativeAIEmbeddings

embeddings = GoogleGenerativeAIEmbeddings(model="gemini-embedding-001")
sentences = [
    "How many days of paid leave do employees get?",
    "Staff receive 24 days of annual vacation.",
    "Attention maps a query and key-value pairs to an output.",
]
vectors = np.array(embeddings.embed_documents(sentences))
print(vectors.shape)
unit = vectors / np.linalg.norm(vectors, axis=1, keepdims=True)
print(np.round(unit @ unit.T, 3))

Reading the similarity matrix

  • The shape (3, 3072) says three texts, each a vector of 3072 numbers, the size this model returns by default.
  • The first two sentences score 0.891. One is a question about paid leave and the other an answer about vacation days, with few words in common, and they are the closest pair in the matrix.
  • The attention sentence scores 0.707 and 0.687 against them, lower, as it is about a different subject.
  • Unrelated text still scores high. Even the unrelated pairs are near 0.7, so compare scores with each other, not with 0 or 1.

Storing chunks in a vector store

A vector store: collection, persistence, ids and metadata · from the Complete Agentic AI Course video · 6:11:09 to 6:14:41

The video wraps a Chroma database in a class: a client with a persistent folder, a collection to hold the records, and an add method that needs four things for each chunk, an id, the text, its metadata and its embedding. The code below keeps the same four things but uses LangChain's vector store interface, so the same few calls work on any store. InMemoryVectorStore keeps everything in Python memory and ships with langchain-core.

Building the store from chunks

python
from langchain_core.vectorstores import InMemoryVectorStore

store = InMemoryVectorStore.from_documents(chunks, embeddings)   # embeds every chunk, keeps the vectors

Searching with a score

python
results = store.similarity_search_with_score("What is scaled dot-product attention?", k=3)
for doc, score in results:                  # the k closest chunks, best first
    print(round(score, 3), doc.metadata["source"], doc.metadata["page"])

This example uses two of the papers and four of their pages, so that the whole index stays a few thousand tokens. The later RAG lessons start from the same chunks.

ExampleAPI keyPICK keeps pages 3 and 4 of the Transformer paper and pages 0 and 1 of the QZhou report; run on gemini-embedding-001
import os, urllib.request
from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_google_genai import GoogleGenerativeAIEmbeddings
from langchain_core.vectorstores import InMemoryVectorStore

PAPERS = {"attention": "1706.03762", "qzhou": "2508.21632"}
PICK = {"attention": [3, 4], "qzhou": [0, 1]}
pages = []
for name, arxiv_id in PAPERS.items():
    if not os.path.exists(f"{name}.pdf"):
        urllib.request.urlretrieve(f"https://arxiv.org/pdf/{arxiv_id}", f"{name}.pdf")
    docs = PyPDFLoader(f"{name}.pdf").load()
    pages += [docs[i] for i in PICK[name]]
chunks = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200).split_documents(pages)
for i, c in enumerate(chunks):
    c.id = f"c{i}"
print(len(pages), "pages ->", len(chunks), "chunks")

store = InMemoryVectorStore.from_documents(chunks, GoogleGenerativeAIEmbeddings(model="gemini-embedding-001"))
for q in ["What is scaled dot-product attention?", "Which model is QZhou-Embedding built on?"]:
    print(q)
    for doc, score in store.similarity_search_with_score(q, k=3):
        print("  ", round(score, 3), doc.id, doc.metadata["source"], doc.metadata["page"], repr(doc.page_content[:40]))

Reading the search results

  • Four pages became 15 chunks, with the ids c0 to c14 from the loop.
  • Each result row is the score, the chunk id, the file, the page index and the start of the text. The scores are cosine similarities, so the best chunk has the highest number and comes first.
  • The attention question returned c0, c1 and c2, all from page 3 of attention.pdf, with scores 0.788, 0.779 and 0.77. Neighbouring chunks share text through the overlap, so three results can say nearly the same thing.
  • The QZhou question returned only chunks of qzhou.pdf. The top one, c7 with 0.797, is the abstract, which says the model is built on Qwen2.5-7B-Instruct.

Stable ids and saving to disk with FAISS

A store needs an id for every record, and the id must be the same every time the same chunk is added; with random ids, running the indexing code twice adds a second copy of every chunk. The loop c.id = f"c{i}" in the example above gives each chunk a stable id, and a store that is given the same id twice can then refuse or replace instead of piling up copies.

FAISS is a similarity-search library from Meta that runs in your process and saves to a folder. LangChain wraps it as FAISS in langchain-community (pip install faiss-cpu). It stores vectors in a flat index and measures squared L2 distance (the straight-line gap between two vectors, squared), so a smaller number means closer; that is the opposite direction from a cosine score.

Building, saving and loading a FAISS index

python
from langchain_community.vectorstores import FAISS

store = FAISS.from_documents(chunks, embeddings, ids=[c.id for c in chunks])
store.save_local("faiss_demo")                     # writes index.faiss and index.pkl
again = FAISS.load_local("faiss_demo", embeddings, allow_dangerous_deserialization=True)
ExampleAPI keyStarts from the chunks of the example above; run on faiss-cpu 1.15.1
from langchain_community.vectorstores import FAISS
from langchain_google_genai import GoogleGenerativeAIEmbeddings

embeddings = GoogleGenerativeAIEmbeddings(model="gemini-embedding-001")
store = FAISS.from_documents(chunks, embeddings, ids=[c.id for c in chunks])
print(store.index.ntotal, "vectors of", store.index.d, "numbers,", type(store.index).__name__)
for doc, distance in store.similarity_search_with_score("What is scaled dot-product attention?", k=3):
    print(round(float(distance), 3), doc.id, doc.metadata["source"], doc.metadata["page"])

store.save_local("faiss_demo")
again = FAISS.load_local("faiss_demo", embeddings, allow_dangerous_deserialization=True)
print("loaded:", again.index.ntotal, "vectors")
try:
    again.add_documents(chunks[:3], ids=[c.id for c in chunks[:3]])
except ValueError as err:
    print("ValueError:", str(err)[:60])

Reading the FAISS run

  • FAISS holds 15 vectors of 3072 numbers in an IndexFlatL2, a flat index that compares the question with every vector.
  • The top three are c0, c1 and c2 again, as with cosine, now as distances 0.423, 0.442 and 0.46. The smallest number is the best match. The vectors have length 1, and for such vectors the squared distance equals 2 minus 2 times the cosine similarity: 2 − 2 × 0.788 = 0.424, 2 − 2 × 0.779 = 0.442 and 2 − 2 × 0.77 = 0.46, the three distances above up to rounding.
  • Saving and loading round-trips. The loaded store has 15 vectors again. save_local writes two files, index.faiss for the vectors and index.pkl for the texts and metadata.
  • Re-adding the same ids is refused. FAISS raises a ValueError that names c0, c1 and c2, so a re-run that repeats ids shows up as an error and not as silent copies.

Cosine similarity vs L2 distance vs dot product

Cosine similaritySquared L2 distanceDot product
MeasuresThe angle between two vectorsThe squared straight-line gap between two vectorsAngle and length together
Closer meansA higher numberA lower numberA higher number
Where it shows up hereInMemoryVectorStore scoresFAISS default scoresEquals cosine when vectors have length 1

Where you use embeddings and vector stores

  • Semantic search. Finding the chunk about "vacation" for a question about "paid leave", with no shared word.
  • RAG retrieval. The vector store is the box in the middle of the pipeline that every question searches.
  • Deduplication and clustering. Texts whose vectors are very close are near-copies, which helps to clean a corpus before it is indexed.
Watch out. A score only means something inside one store and one metric. InMemoryVectorStore returns cosine similarity (higher is closer), FAISS returns squared L2 distance (lower is closer), and Chroma's default distance is squared L2, so 1 - distance is a cosine similarity only when the collection is created with the cosine space. Embed questions with the same model that embedded the chunks; vectors from two different models cannot be compared.
Try it yourself
  • Add a fourth sentence to the first example that mentions leave in other words, such as "Time off per year is 24 days", and compare its row of the matrix with the first sentence.
  • Change k=3 to k=6 in the search and read where the scores start to drop.
  • Search for a question that is not in the papers, such as "Who won the 2022 World Cup?", and look at the top score next to the one for a question the papers answer.

Slow is fine. Stopping is the only problem.