DeepEvaldeepeval 4.2.8 · Python 3.10+
Dashboard
0%
1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
34 small wins to finish your pathNext lesson →

TechNest RAG app

A RAG app is a program that retrieves the knowledge-base entries closest to a question and passes them to an LLM, which writes the answer from those entries only.

Last updated: 05 Oct, 2026 · DeepEval 4.2.8

An evaluation needs an app to evaluate. This lesson builds the TechNest support bot as one file, technest.py, so every later lesson can ask it questions and score what it says.

The TechNest app and its knowledge base · from the Production RAG Live Marathon · 2:44:03 to 2:47:04

The video moves from a plain RAG chatbot, where you cannot see on what basis it answered, to the same kind of bot with an evaluation pipeline around it. That app asks for a Groq key for its answers, a second Groq key for the judge and a Gemini key for embeddings. Its knowledge base belongs to a company that sells tech devices online: each entry has an id, a category, a title and the content, covering products, shipping and return policies, and order tracking.

The retrieve and generate API

python
contexts = retrieve(question, top_k=3)   # the 3 catalog entries closest to the question
response = generate(question, contexts)  # the LLM answers from those entries only
response, contexts = answer(question)    # both steps together

The knowledge base

Save the video's catalog as catalog.json: 8 products, 4 policies and 3 FAQ entries. This is the file the bot searches.

json
[
  {
    "id": "prod_001",
    "category": "product",
    "title": "ProBook X1 Laptop",
    "content": "The TechNest ProBook X1 is a 14-inch laptop featuring an Intel Core i7-13th Gen processor, 16GB DDR5 RAM, and a 512GB NVMe SSD. It has a battery life of 12 hours, weighs 1.4kg, and comes with a backlit keyboard. Price: $1,299. Includes a 2-year manufacturer warranty."
  },
  {
    "id": "prod_002",
    "category": "product",
    "title": "PixelPhone 15",
    "content": "The TechNest PixelPhone 15 is a 6.7-inch AMOLED smartphone with a 50MP triple camera system, 8GB RAM, 256GB storage, and a 5,000mAh battery supporting 65W fast charging. Available in Midnight Black and Arctic White. Price: $899. Includes a 1-year warranty."
  },
  {
    "id": "prod_003",
    "category": "product",
    "title": "SoundPods Pro",
    "content": "The TechNest SoundPods Pro are true wireless earbuds with active noise cancellation (ANC), 8 hours of playback per charge plus 24 hours with the case, and IPX4 water resistance. They connect via Bluetooth 5.3 and support multipoint pairing with two devices simultaneously. Price: $149."
  },
  {
    "id": "prod_004",
    "category": "product",
    "title": "UltraTab S2 Tablet",
    "content": "The TechNest UltraTab S2 is a 11-inch tablet powered by a Snapdragon 870 processor with 8GB RAM and 128GB storage expandable via microSD. It features a 120Hz display, a 7,500mAh battery, and supports the TechNest Stylus Pen sold separately. Price: $549. Includes a 1-year warranty."
  },
  {
    "id": "prod_005",
    "category": "product",
    "title": "SmartWatch X",
    "content": "The TechNest SmartWatch X features continuous heart rate monitoring, SpO2 tracking, GPS, and 7-day battery life. It is water-resistant up to 50 metres. Compatible with both Android and iOS. Price: $299. Includes a 1-year warranty and a free extra silicone band."
  },
  {
    "id": "prod_006",
    "category": "product",
    "title": "ProCam 4K Action Camera",
    "content": "The TechNest ProCam 4K shoots 4K video at 60fps and 20MP photos. It is waterproof up to 10 metres without a case, has built-in image stabilisation (EIS), and includes a touch screen. Battery life is 90 minutes of 4K recording. Price: $229. Includes a 1-year warranty."
  },
  {
    "id": "prod_007",
    "category": "product",
    "title": "BassBuds Max Headphones",
    "content": "The TechNest BassBuds Max are over-ear wireless headphones with 40-hour battery life, hybrid active noise cancellation, and a premium 40mm driver for deep bass. They fold flat for travel and include a carrying case. Price: $199. Compatible with all Bluetooth devices."
  },
  {
    "id": "prod_008",
    "category": "product",
    "title": "SoundBar 360",
    "content": "The TechNest SoundBar 360 is a 2.1 soundbar with a 120W output, built-in subwoofer, Dolby Atmos support, and HDMI ARC connectivity. It also supports Bluetooth streaming and has an optical audio input. Dimensions: 90cm wide. Price: $349. Includes a 2-year warranty."
  },
  {
    "id": "policy_001",
    "category": "policy",
    "title": "Return Policy",
    "content": "TechNest accepts returns within 30 days of the original purchase date. Items must be in their original packaging with all accessories included. Customers are responsible for return shipping costs unless the item arrives defective or damaged. Refunds are processed within 5 to 7 business days of receiving the returned item. Digital downloads and opened software are non-refundable."
  },
  {
    "id": "policy_002",
    "category": "policy",
    "title": "Shipping Policy",
    "content": "TechNest offers free standard shipping on all orders over $50 within the continental US. Standard shipping takes 3 to 5 business days. Expedited shipping (1 to 2 business days) is available for $9.99. Same-day delivery is available in select cities for $19.99. Orders placed before 2pm local time are dispatched the same day."
  },
  {
    "id": "policy_003",
    "category": "policy",
    "title": "Warranty Policy",
    "content": "All TechNest products include a minimum 1-year manufacturer warranty covering defects in materials and workmanship. The ProBook X1 and SoundBar 360 include a 2-year warranty. Warranty does not cover physical damage, water damage (unless the product is rated waterproof), or damage from unauthorised modifications. To make a warranty claim, contact support@technest.com with your order number and a description of the issue."
  },
  {
    "id": "policy_004",
    "category": "policy",
    "title": "Payment Policy",
    "content": "TechNest accepts Visa, Mastercard, American Express, PayPal, and Apple Pay. All transactions are encrypted using 256-bit SSL. Buy Now Pay Later is available via Klarna for orders over $100, with 0% interest for 3 monthly instalments. TechNest does not store full card details — payments are processed securely by Stripe."
  },
  {
    "id": "faq_001",
    "category": "faq",
    "title": "Order Tracking",
    "content": "To track your order, visit technest.com/orders and enter your order number and email address. A shipping confirmation email with a tracking link is sent within 24 hours of dispatch. If you have not received your tracking email after 48 hours, check your spam folder or contact support@technest.com."
  },
  {
    "id": "faq_002",
    "category": "faq",
    "title": "International Shipping",
    "content": "TechNest ships to over 40 countries. International shipping rates start at $14.99 and delivery takes 7 to 14 business days. Import duties and taxes are the responsibility of the customer and are not included in the product price. Free shipping promotions apply to US orders only."
  },
  {
    "id": "faq_003",
    "category": "faq",
    "title": "Bulk and Business Orders",
    "content": "TechNest offers volume discounts for businesses purchasing 10 or more units of any single product. Discounts range from 10% for 10 to 49 units up to 25% for 100 or more units. Contact business@technest.com with your requirements for a custom quote. A dedicated account manager is assigned for orders over $10,000."
  }
]

Matching words between question and entry

words lowercases a text, splits it into words and drops the ones that appear in almost every question, such as what, is and TechNest. What is left are the words that say what the question is about.

python
SKIP = {"a", "an", "and", "are", "can", "do", "does", "for", "how", "i", "in", "is", "it",
        "long", "much", "my", "of", "on", "s", "technest", "the", "to", "what", "with", "you", "your"}


def words(text):
    """The words in a text that carry meaning, in lower case."""
    return {w for w in re.findall(r"[a-z0-9]+", text.lower()) if w not in SKIP}

Ranking the catalog

python
def retrieve(question, top_k=3):
    asked = words(question)
    ranked = sorted(CATALOG, key=lambda item: len(asked & words(item["title"] + " " + item["content"])), reverse=True)
    return [item["content"] for item in ranked[:top_k]]

Each entry scores the number of question words it shares, and the top_k highest come back, best first. Ties keep the catalog's order. This ranking is easy to read, and it makes mistakes a real search also makes, which gives the retrieval metrics later in the course something to find.

Generating the answer with Groq

The system prompt is the video's, word for word, plus one last line that keeps answers short.

python
def generate(question, contexts):
    context_block = "\n\n".join(f"[{i+1}] {c}" for i, c in enumerate(contexts))
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": f"Context:\n{context_block}\n\nCustomer question: {question}"},
    ]
    response = groq.chat.completions.create(model=CHAT_MODEL, messages=messages, temperature=0)
    return response.choices[0].message.content.strip()

The numbered entries go in the user message, and the prompt tells the model to answer from them only and to say so when they are not enough.

The whole technest.py file

Put the pieces together in technest.py, next to catalog.json. Later lessons import answer, retrieve and generate from it.

python
import json
import os
import re

from openai import OpenAI

groq = OpenAI(api_key=os.environ["GROQ_API_KEY"], base_url="https://api.groq.com/openai/v1")
CHAT_MODEL = "qwen/qwen3.8-27b"

SYSTEM_PROMPT = """You are a helpful customer support assistant for TechNest, an online electronics store.
Answer the customer's question using ONLY the information provided in the context below.
If the context does not contain enough information to answer fully, say so honestly.
Keep your answer concise, factual, and friendly. Do not invent any details not present in the context.
Reply in two or three plain sentences, with no lists or tables."""

with open("catalog.json", encoding="utf-8") as f:
    CATALOG = json.load(f)

SKIP = {"a", "an", "and", "are", "can", "do", "does", "for", "how", "i", "in", "is", "it",
        "long", "much", "my", "of", "on", "s", "technest", "the", "to", "what", "with", "you", "your"}


def words(text):
    """The words in a text that carry meaning, in lower case."""
    return {w for w in re.findall(r"[a-z0-9]+", text.lower()) if w not in SKIP}


def retrieve(question, top_k=3):
    asked = words(question)
    ranked = sorted(CATALOG, key=lambda item: len(asked & words(item["title"] + " " + item["content"])), reverse=True)
    return [item["content"] for item in ranked[:top_k]]


def generate(question, contexts):
    context_block = "\n\n".join(f"[{i+1}] {c}" for i, c in enumerate(contexts))
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": f"Context:\n{context_block}\n\nCustomer question: {question}"},
    ]
    response = groq.chat.completions.create(model=CHAT_MODEL, messages=messages, temperature=0)
    return response.choices[0].message.content.strip()


def answer(question, top_k=3):
    contexts = retrieve(question, top_k)
    return generate(question, contexts), contexts

This file differs from the video's app in two ways. The video's app finds entries with Gemini embeddings; this file matches words, so the whole course runs on one Groq key. And the video's repository answers with llama-3.3-70b-versatile, which Groq has since retired; this file answers with qwen/qwen3.8-27b, a different model family from the judge the course builds later, so the judge is not grading its own writing.

Asking the bot about the return policy

ExampleAPI keytechnest.py and catalog.json in the same folder
from technest import answer

response, contexts = answer("What is TechNest's return policy?")
for number, chunk in enumerate(contexts, 1):
    print(f"chunk {number}: {chunk[:70]}...")
print()
print(response)

What the bot retrieved and said

  • chunk 1 is the return policy, the entry the question needs, ranked first.
  • chunks 2 and 3 are the shipping and warranty policies. They share the word policy with the question but do not answer it; whether they hurt is what the retrieval metrics measure.
  • The answer is written from chunk 1. Whether every sentence in it is backed by a chunk is what Faithfulness checks.

Word matching vs embeddings

Shared words (this file)Embeddings (the video's app)
FindsEntries that use the question's wordsEntries with a similar meaning
NeedsNothing beyond PythonAn embedding model and its key
MissesQuestions phrased in other wordsRare names and exact numbers, sometimes
Same for evaluation?Yes: both return a ranked list of chunks to scoreYes

When to build the app this way

  • When you want retrieval and generation as separate functions, so an evaluation can keep what each one returned.
  • When the knowledge base is small enough to hold in memory. For a large one, swap retrieve for a vector store and keep the same return value.
Watch out. Keep the retrieved chunks, not only the answer. An app that returns only text makes it impossible to tell later whether a wrong answer came from the search or from the model.
Try it yourself
  • Call retrieve("How long is the battery life on the SoundPods Pro?"): the ProBook X1 comes back first, because it shares battery and life while the SoundPods entry shares soundpods and pro, and ties keep the catalog's order. The retrieval metrics catch this kind of miss.
  • Ask answer("Do you ship to Canada?") and read how the bot answers when no chunk mentions Canada.
  • Call retrieve("What is the price of the PixelPhone 15?", top_k=1) and print the single chunk.
PreviousLLM evaluation

Slow is fine. Stopping is the only problem.