A product manager edited a prompt directly in the prompt-management UI, and it broke a production workflow. How do you fix the process without slowing everyone down?

30-second answerSay your answer out loud first, then reveal.

Keep prompt editing accessible to non-engineers, but route changes through the same safety rails as code: version history, a review or approval step for production prompts, automatic eval runs on every change (with results shown in the UI), staged rollout via flags, and one-click rollback. Separate draft/staging prompts from production. Add ownership and change notifications. The goal is a fast path with guardrails, not banning PMs from prompts.

Process design

Environments for prompts: edits happen in draft → evaluated automatically → promoted to staging → production with approval.
Automatic eval on save: the UI shows golden-set results vs the current production version (pass/fail per criterion, examples that flipped).
Approvals: owners for each prompt (code-owner style); high-risk features need a second reviewer.
Progressive rollout: new prompt versions start at 5% via a flag, with automatic rollback on metric regressions.
Contracts: structured-output prompts have schema tests (a PM's wording change can't break JSON parsing silently).
Audit and notifications: a changelog in a team channel ("Prompt claims_summary v15 promoted by X; evals: +2% quality").
Rollback: one click to the previous version.

Post-incident: a blameless review. The system allowed an unsafe change; the person isn't the root cause.

PreviousYour outputs changed in style and quality overnight with no deploy. You suspect the provider updated the model. How do you confirm it and respond?

Every expert started right here.

A product manager edited a prompt directly in the prompt-management UI, and it broke a production workflow. How do you fix the process without slowing everyone down?

Process design

Related