Curious builder0 XP earned · 300 to level 2

0 daysFinish a lesson to begin

Badge collection0 of 6 unlocked

19 small wins to finish your path

The triage service, end to end

Lesson 0 promised a service that takes support tickets, asks a model API to sort each one, checks the answer and replies. Here it runs, with five tickets sent at once.

The folder holds model_api.py with the key check from lesson 10, service.py from lessons 15 and 16, and tickets.json from Python for AI. One change to the service first.

When the model API is down

Exampleservice.py, inside triage

    try:
        response = model.post("/v1/chat/completions", json=body)
        response.raise_for_status()
    except httpx2.HTTPError:
        raise HTTPException(status_code=503, detail="the model API is not answering")

httpx2.HTTPError covers both kinds of failure from lesson 12: no response, and an error status from raise_for_status. Either way the client now gets 503, service unavailable, a code that says trying again later may work, instead of a 500 crash.

The client

Examplesend.py, part 1

import asyncio
import json

import httpx2


async def send(client, ticket):
    response = await client.post("/triage", json={"id": ticket["id"], "text": ticket["text"]})
    if response.status_code == 200:
        return response.json()
    return {"id": ticket["id"], "error": response.status_code}

Examplesend.py, part 2

async def main():
    with open("tickets.json") as f:
        tickets = json.load(f)
    headers = {"X-API-Key": "team-key-1"}
    async with httpx2.AsyncClient(base_url="http://127.0.0.1:8100", headers=headers) as client:
        results = await asyncio.gather(*[send(client, ticket) for ticket in tickets])
    for row in results:
        print(row)


asyncio.run(main())

The async batch from lesson 13, pointed at the service's real address. A ticket that is not sorted is kept with its status code instead of stopping the batch.

Run it

Example

export MODEL_API_KEY=sk-local-123
uvicorn model_api:app --port 8101 --log-level warning &
uvicorn service:app --port 8100 --log-level warning &
sleep 3
python send.py
kill %1 %2

Five tickets, sent together, each checked. Ticket 4 is reported as 502, so whoever runs this knows exactly which ticket a person should sort.

And with the model API stopped

Example

export MODEL_API_KEY=sk-local-123
uvicorn service:app --port 8100 --log-level warning &
sleep 3
python send.py
kill %1

Every ticket gets 503. The service stayed up, said what was wrong, and the client kept every ticket to send again.

Where each piece came from

The service, by lesson

Things to add

Try it yourself

Add the ask retry from lesson 11 inside triage, and turn the model API's rate limit back on.
Add a GET /health endpoint to the service that also checks the model API answers.
Write a test that makes fake_model_client raise httpx2.ConnectError and checks for 503.

What this course left out

Topic	What it is for
OAuth and user logins	Signing in people rather than checking one key per team; FastAPI's security guide covers OAuth2 and JWT tokens.
CORS	Letting a web page on another domain call your API from the browser.
Databases	Storing tickets and results so they survive a restart.
Background tasks and queues	Answering at once and doing slow model work afterwards.
Deployment	Running the service in a container on a cloud host, behind HTTPS.
Provider SDKs	The OpenAI and Anthropic Python packages wrap these same HTTP calls, retries and streaming for you.

PreviousTesting the service with pytest

Slow is fine. Stopping is the only problem.