Deployed agent platform: live, traced and affordable
A pilot is not production. This project is everything that turns a working agent into one a team can depend on.
The brief
The operations team loved the workflow agent pilot. Now 500 employees will use it, the security team has questions, and finance wants to know what it costs.
What you will build
- The agent served behind an API with streaming
- Real models, with a fallback when the main one fails
- Tracing and a cost dashboard
- CI that tests and reviews every change
- Locked-down permissions for anything the agent runs
Requirements
| Requirement | Done when |
|---|---|
| Serving | Runs in the cloud behind an authenticated API |
| Fallback | Turning off the main model switches to the backup |
| Observability | Every run is traced; daily cost is visible |
| CI | Every pull request runs tests, evals and a review |
| Security | A one-page answer to the security questionnaire |
What to learn it from
- LangGraph: real models and a server
- LiteLLM: a gateway with fallback models and cost tracking
- Claude Code: agents in CI
- OpenAI Codex: sandbox and security model
- Promptfoo: evals that fail the build
Milestones
- Containerise the agent
- Deploy it behind an API
- Add tracing and the cost view
- Add the fallback model and test it
- Add CI with evals and review
- Write the runbook and the security answer
How it is judged
Turn off the main model provider during a demo. Users should notice nothing but a note in the traces.
Stretch goals
- Autoscaling under load
- Per-team budgets
- Blue-green releases with automatic rollback
Before you call it done
Worth remembering
- Someone else can deploy it from the runbook
- Failure was tested, not assumed
- Cost per run is known
Every expert started right here.