Your customer-support agent issued a refund that violated policy. What happened, and how do you prevent it?

Step 1 — Contain

Pause or limit the refund tool if there's a risk of more incidents, and reverse the refund if appropriate.

Step 2 — Root cause from traces

• Was the refund policy in the prompt at all, and was it clear ("within 30 days of delivery")?
• Did the agent have the facts (order date, delivery status), or did it assume them?
• Did the user manipulate it ("my manager already approved this")? Was there injected text in an email or ticket?
• Did the tool accept anything the model sent?

Step 3 — Fix in layers

1. Hard enforcement in the tool (most important): issue_refund checks the order exists, is within the window, amount ≤ order total, amount ≤ auto-approve limit, and the customer's refund count. The model can request the refund; code decides whether it is allowed.
2. Approval workflow: above ₹X or outside the standard window → create a pending refund for human approval.
3. Prompt improvements: clarify the policy, and tell the agent to verify facts using tools before acting.
4. Separation: the policy engine is a deterministic service, not LLM judgement.
5. Monitoring: alerts on refund rate, amount anomalies, and repeated refunds per user.

Step 4 — Regression tests

Add this conversation, plus adversarial variants (social engineering, fake approvals), to the eval set.

Related

• Previous: Q25. What is prompt injection in the context of agents? Direct vs indirect?
• Next: Q27. How do you reduce an agent's latency?