Where do guardrails fit in an AI system, and what do they check?

30-second answerSay your answer out loud first, then reveal.

Guardrails are checks around the model. Input guardrails run before the LLM: authentication context, PII detection/redaction, prompt-injection and jailbreak detection, topic/abuse filters, size limits. Output guardrails run after: harmful content, PII leakage, policy compliance, schema validation, groundedness/citation checks, brand tone. Action guardrails sit at tool boundaries: permissions, approvals, limits. Use fast classifiers and rules, keep latency low, and log every decision.

User input passes input guardrails (or gets a safe refusal), then the LLM and tools, with action guardrails on tool calls and output guardrails that either pass the response or trigger regeneration, fallback or human review.

Implementation options: rules and regexes (fast, for PII and keywords), small classifiers (toxicity, injection), LLM-based checks (nuanced policy, slower), frameworks (NeMo Guardrails, Guardrails AI, cloud content-safety APIs).

Design considerations

Latency: run input checks in parallel with retrieval; for streaming, check output in chunks or with a buffer (trade-off between safety and speed).
False positives: over-blocking frustrates users, so measure both block rate and miss rate.
Fail-open vs fail-closed: if the guardrail service is down, do you block (safe) or allow (available)? It depends on risk.
Audit logs of blocked content for tuning and compliance.

PreviousHow do you handle provider rate limits and quotas in an LLM system?

Every expert started right here.

Where do guardrails fit in an AI system, and what do they check?

Design considerations

Related