Policies in a store that stays

InMemoryVectorStore builds its index at every start. A real store keeps it on disk, and the swap is the constructor plus two details that matter.

Chroma is a vector database that runs inside your own process and writes to a folder. search.py changes at the top; the tool below it is untouched.

from langchain.tools import tool
from langchain_chroma import Chroma
from policies import chunks
from word_embeddings import WordEmbeddings

store = Chroma(collection_name="policies", embedding_function=WordEmbeddings(),
               persist_directory="./policy_store",
               collection_metadata={"hnsw:space": "cosine"})
if not store.get()["ids"]:
    store.add_documents(chunks)


@tool
def search_policies(query: str) -> str:
    """Search the shop's policies on refunds, shipping and accounts."""
    found = [doc for doc, score in store.similarity_search_with_relevance_scores(query, k=2)
             if score >= 0.3]
    if not found:
        return "No policy covers this."
    return "\n".join(f"[{doc.metadata['source']}] {doc.page_content}" for doc in found)

persist_directory is the folder it writes to, and get() returns what is already in there, so the chunks are added once rather than on every start.

The score is not the same number

A store also decides what its scores mean. Chroma's default answers with a distance, where lower is closer, and a plain similarity_search_with_score shows it.

from langchain_chroma import Chroma
from policies import chunks
from word_embeddings import WordEmbeddings

plain = Chroma(collection_name="plain-store", embedding_function=WordEmbeddings())
plain.add_documents(chunks)
for doc, score in plain.similarity_search_with_score("do you sell gift cards", k=2):
    print(round(score, 2), doc.metadata["source"])

Those are distances: 11 and 12, for a question no policy covers. A score >= 0.3 cut would have let both through and the desk would have quoted the shipping policy at a customer asking about gift cards. Two changes keep the old behaviour: hnsw:space asks for cosine, and similarity_search_with_relevance_scores returns 0 to 1 with higher meaning closer.

The desk on the real store

from chat import say
from search import store

say("ravi", "How long does a refund take?", "ravi-9")
say("ravi", "Do you sell gift cards?", "ravi-9")
print(len(store.get()["ids"]), "chunks kept in ./policy_store")

The refund policy is found and the gift card question is still refused, so the cut survived the move. The five chunks now sit in ./policy_store, and the next start reads them instead of splitting the documents again.

The embedder is the same swap

WordEmbeddings is two methods, and a hosted embedder is the same two. Installing langchain-openai and changing the argument is the whole move.

from langchain_openai import OpenAIEmbeddings

store = Chroma(collection_name="policies", embedding_function=OpenAIEmbeddings(),
               persist_directory="./policy_store",
               collection_metadata={"hnsw:space": "cosine"})

It reads OPENAI_API_KEY and charges for what it embeds. One thing does change, though. Vectors from one embedder mean nothing to another, so the folder has to be deleted and the chunks added again, and the 0.3 cut is worth re-checking against real vectors, which sit much closer together than word counts do.