LLMOpsThe LLMOps lifecycle · MLflow 3 · FastAPI · Docker · AWS/Azure
Dashboard
0%
1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
41 small wins to finish your pathNext lesson →

A production project layout

A production project layout is the set of folders an LLM app grows into once it leaves a notebook: one place for configuration, one for prompts, one for logging, one for the app's own errors, and a single entry point that wires them together.

Last updated: 15 Oct, 2026 · LLMOps

Part 3 ended with the pieces of an LLM app chosen: a model, its tools, and a data pipeline. Those pieces still live in a notebook, where every value is a literal and every helper is a cell. This part turns them into a project you can run, log, test and deploy. The shape used here is the one from the Day 1 to 3 Agentic AI project, the AI_Trip_Planner repository, read as a template for structure rather than for its travel-agent logic.

From one notebook to a tree of folders

A notebook keeps everything in reach, which is what makes it good for trying an idea and bad for running one. A model name buried in a cell cannot be changed without editing code; a prompt copied into three cells drifts out of step; an error prints a stack trace no operator can act on. The project layout gives each of those concerns its own home, so a change to one does not reach into the others.

text
ai-trip-planner/
|-- agent/
|   `-- agentic_workflow.py    the graph that wires the model to its tools
|-- config/
|   `-- config.yaml            model names and settings, kept out of code
|-- prompt_library/
|   `-- prompt.py              the system prompt, written down in one place
|-- tools/                     one file per tool the agent can call
|-- utils/
|   `-- config_loader.py       reads config.yaml into a plain dict
|-- logger/                    one configured logger for the whole app
|-- exception/                 the app's own exception type
|-- notebook/
|   `-- experiments.ipynb      throwaway exploration, kept out of the app
|-- main.py                    the FastAPI entry point
|-- requirements.txt           the dependency list
|-- setup.py                   makes the folders an installable package
`-- pyproject.toml             project metadata

What each folder is for

  • config/ holds a config.yaml of model names and settings. The app reads it at start-up, so the same code runs against a different model by editing a file, not the source.
  • prompt_library/ keeps every prompt in one module. In the repo this is prompt.py with a single SYSTEM_PROMPT, so a wording change happens in one spot and is reviewed like any other code.
  • logger/ configures one logger the whole app shares, so every component writes the same kind of line to the same place.
  • exception/ defines the app's own exception type, so a failure carries the context that caused it instead of a bare message.
  • utils/ holds small helpers such as config_loader.py, the function that reads the YAML into a dict.
  • agent/ is the application logic: agentic_workflow.py builds the graph that binds the model to its tools. tools/ is one file per tool it can call.
  • main.py is the entry point, a FastAPI app that exposes the assistant over HTTP. setup.py and pyproject.toml make the whole thing an installable, versioned package.
A production project layout. At the top, main.py, the entry point that exposes the assistant over HTTP with FastAPI, with arrows down to six folders that each own one concern: config for model names and settings read at start-up, prompt_library for every prompt named in one place, logger for the one logger the app shares, exception for the app's own error type carrying context, utils for small helpers such as the config loader, and agent for the application logic binding the model to its tools. A strip underneath shows setup.py, pyproject.toml and requirements.txt making it installable.

A notebook against a project

A notebookA production project
ConfigurationLiterals in a cellA config.yaml read at start-up
PromptsCopied where neededOne prompt library module
FailuresA raw stack traceThe app's own exception with context
Run itCell by cell, by handOne entry point, under a server
Ship itNot installableA versioned package

Where this layout earns its keep

  • A handover, where a second engineer can find the model setting, the prompt and the entry point without reading every cell.
  • A deployment, where a server imports main.py and a container installs the package, neither of which a notebook supports.
  • A change that must be reviewed, such as a new prompt or a model swap, which lands as a small diff in one file.
Watch out. The point of the layout is that each folder owns one concern, so resist putting a prompt string in main.py or a model name in the agent code. Once a value leaks out of its folder, the benefit of having the folder is gone, and the next person looks in two places instead of one.
Try it yourself
  • Clone the AI_Trip_Planner repository and match each top-level folder to the concern it owns.
  • List every place a model name or a prompt appears in one of your own notebooks, and decide which folder each would move to.
  • Open agent/agentic_workflow.py and note that it imports the prompt and the model loader rather than defining them inline.
PreviousGiving the model tools

Slow is fine. Stopping is the only problem.