Policy desk for customers and staff

Put the course together: one set of policies answered for customers and staff, permissions in the retriever, sourced answers, an agent for orders, and tests.

The project uses desk.py, policies.py and test_desk.py from lesson 17, and shop_chat.py from lesson 11.

from desk import build_desk
from policies import POLICIES
from shop_chat import ShopChat

customer_desk = build_desk(POLICIES, ShopChat(), audience="customer")
staff_desk = build_desk(POLICIES, ShopChat(), audience="staff")

for desk, who in [(customer_desk, "customer"), (staff_desk, "staff")]:
    for question in ["who can approve refunds", "when do parcels ship"]:
        answer = desk.run(question=question)["answers"][0]
        print(f"{who:8} {question:26} -> {answer.data}")

python app.py

The same question gives different answers by audience. For a customer, the staff rule is filtered out before ranking, so the best remaining match is the refunds document. For staff, the staff rule is found. Parcels are public, so both see the same answer.

Adding order lookups

desk = build_desk(POLICIES, ShopChat(), audience="customer")
policy_tool = ComponentTool(component=desk, name="search_policies", description="Answer questions about shop policies.")
agent = Agent(chat_generator=ShopChat(), tools=[lookup_order, policy_tool])

reply = agent.run(messages=[ChatMessage.from_user("Where is A-1002?")])["messages"][-1]
print(reply.text)
print([tool.name for tool in agent.tools])

ComponentTool turned the whole desk into a tool. The stand-in always calls the first tool for an order id, so this ticket went to lookup_order; a real model would read both descriptions and call search_policies for policy questions.

The desk's tests

pytest -q

Switching the desk to OpenAI

Pass OpenAIChatGenerator(model="gpt-4.1-mini") to build_desk in app.py and set OPENAI_API_KEY. The tests keep the stand-ins, because the permission rule and the sources do not depend on the model. Measure retrieval with lesson 14's evaluators on real customer questions before changing splitting or retrieval.

Haystack components not covered