Curious builder0 XP earned · 300 to level 2

0 daysFinish a lesson to begin

Badge collection0 of 6 unlocked

51 small wins to finish your path

Q50HardScenario

You join a company as its first LLMOps engineer. There are 10 ad-hoc LLM applications built by different teams. What do you do in your first 90 days?

30-second answerSay your answer out loud first, then reveal.

Days 1–30: discover and stabilise. Inventory all apps (owners, models, data, costs, risks), add basic observability and cost tracking, fix urgent risks (leaked keys, unpinned models, PII in logs). Days 31–60: standardise. Introduce a gateway, a prompt and config versioning pattern, a minimal eval and CI template, and guardrail baselines, piloted with two willing teams. Days 61–90: scale and govern. Roll out the golden path to the remaining teams, define SLOs and incident processes, set cost budgets, and report wins. Prioritise by risk and value, and earn adoption with helpful tooling, not mandates.

Days 1–30: discover and de-risk

Interview each team; build an inventory: purpose, users, models and versions, data sources, PII exposure, monthly cost, owner.
Quick wins: move API keys into a secrets manager, pin model versions, redact PII in logs, set provider spend limits.
Basic tracing and cost dashboards (even if just via the provider usage APIs at first).
Risk ranking: customer-facing + sensitive data + autonomous actions = top priority.

Days 31–60: build the golden path

LLM gateway (provider abstraction, keys, cost attribution, fallbacks).
Templates: prompt versioning in git, eval harness with a CI gate, standard tracing (OpenTelemetry), guardrail baseline (PII, injection checks).
Pilot with 2 teams; collect feedback; iterate.

Days 61–90: scale and govern

Migrate the remaining apps by priority; office hours and docs.
SLOs for critical apps; on-call and incident runbooks; kill switches.
Budgets and alerts per team; a monthly cost and quality review.
Launch-readiness checklist for new AI features.
Report to leadership: risk reduced, cost visibility, savings achieved, roadmap for the next quarter (fine-tuning pipeline, self-hosting evaluation, etc.).

Interview signal. Pragmatic sequencing (risk first), measurable outcomes, platform thinking, and change management, enabling teams rather than policing them.

Previous: Q49. How do you choose between building and buying each layer of the LLMOps stack?

PreviousHow do you choose between building and buying each layer of the LLMOps stack?

Every expert started right here.

You join a company as its first LLMOps engineer. There are 10 ad-hoc LLM applications built by different teams. What do you do in your first 90 days?

Days 1–30: discover and de-risk

Days 31–60: build the golden path

Days 61–90: scale and govern

Related