Guardrails on every request
A guardrail is a check that the gateway runs on a request or a response, so every application behind the gateway gets the same rule without adding code.
Last updated: 12 Oct, 2026 · LiteLLM 1.105
The admin UI: seeing what the gateway did and the lessons before it set up and watched the gateway. A guardrail changes what it does. Because every call already passes through the gateway, a rule written once there applies to every application, including the ones whose authors never heard of the rule. This lesson writes one guardrail of our own and runs it, and then lists the ready-made ones from the documentation without running them.
Syntax: a guardrail is declared under guardrails in the config, with a name, the guardrail to use and the mode, which says when it runs. The guardrails quick start gives this shape.
guardrails:
- guardrail_name: "my-guard"
litellm_params:
guardrail: <provider-or-your.module.Class>
mode: "pre_call" # when it runs
default_on: true # run it for every requestChoosing when a guardrail runs
The quick start page lists four modes. The table later in this lesson compares them. pre_call runs before the model is called and checks the input. post_call runs after the model answers and sees the input and the output. during_call runs in parallel with the model call, and the response is not returned until the check completes. logging_only scans the logged input and output without changing what the client receives. A list such as [pre_call, post_call] runs several modes.
Writing a guardrail of your own
The class and its hook
A guardrail of your own is a class that extends CustomGuardrail from litellm.integrations.custom_guardrail. Its async_pre_call_hook receives the key's details, the cache, the request body and the call type. The custom guardrail page shows this hook and says returning a changed data rewrites the request. Returning data unchanged lets the request through. The page does not show an HTTPException. Raising one to refuse the request with a status of your choice is what the run below demonstrates.
from fastapi import HTTPException
from litellm.integrations.custom_guardrail import CustomGuardrail
class NoPasswords(CustomGuardrail):
"""Refuses any request that mentions a password."""
async def async_pre_call_hook(self, user_api_key_dict, cache, data, call_type):
for message in data.get("messages", []):
if "password" in str(message.get("content", "")).lower():
raise HTTPException(
status_code=400,
detail={"error": "The desk never handles passwords. Use the reset link."},
)
return dataSwitching it on in the config
guardrail: desk_guardrail.NoPasswords names the file and the class. mode: pre_call runs it before the model. default_on: true applies it to every request. The quick start page says a default-on guardrail runs even when the request names other guardrails, or none. A guardrail without default_on runs only when the request body lists it in a guardrails array.
Running a request through the guardrail
The example writes the guardrail file and a config, starts a real gateway, and sends two requests. The first is harmless and goes to the Groq model. The second mentions a password and should be refused before any model is called.
import subprocess
import time
import urllib.request
from pathlib import Path
import httpx
def start(port=4400, env=None):
proc = subprocess.Popen(
["litellm", "--config", "config.yaml", "--port", str(port)],
stdout=open("gateway.log", "w"), stderr=subprocess.STDOUT, env=env,
)
while True:
try:
urllib.request.urlopen(f"http://127.0.0.1:{port}/health/liveliness")
return proc
except OSError:
time.sleep(1)
Path("desk_guardrail.py").write_text(r'''from fastapi import HTTPException
from litellm.integrations.custom_guardrail import CustomGuardrail
class NoPasswords(CustomGuardrail):
"""Refuses any request that mentions a password."""
async def async_pre_call_hook(self, user_api_key_dict, cache, data, call_type):
for message in data.get("messages", []):
if "password" in str(message.get("content", "")).lower():
raise HTTPException(
status_code=400,
detail={"error": "The desk never handles passwords. Use the reset link."},
)
return data
''')
Path("config.yaml").write_text("""model_list:
- model_name: careful
litellm_params:
model: groq/openai/gpt-oss-120b
api_key: os.environ/GROQ_API_KEY
guardrails:
- guardrail_name: no-passwords
litellm_params:
guardrail: desk_guardrail.NoPasswords
mode: pre_call
default_on: true
general_settings:
master_key: sk-gateway-1234
""")
def ask(text):
r = httpx.post(
"http://127.0.0.1:4400/chat/completions",
headers={"Authorization": "Bearer sk-gateway-1234"},
json={"model": "careful", "max_tokens": 200,
"messages": [{"role": "user", "content": text}]},
timeout=60,
)
body = r.json()
if r.status_code != 200:
return r.status_code, body["error"]["message"]
return r.status_code, body["choices"][0]["message"]["content"]
proc = start()
try:
print(*ask("Reply with one word: ok"))
print(*ask("what is my password"))
finally:
proc.terminate()
proc.wait()200 ok 400 The desk never handles passwords. Use the reset link.
- The first request was answered. It passed the guardrail and the gateway called the Groq model, so its reply is a live model answer and can differ on your run.
- The second request was refused with 400. The message is the text the guardrail raised, carried back to the caller.
- No model saw the second request. The guardrail ran in
pre_callmode, so the refusal came before the provider was called, and no tokens were spent on it.
The ready-made guardrails
The quick start page shows config for several guardrails supplied by other companies and for Presidio. Its examples for Aporia, Cato Networks, Pillar Security and Azure text moderation each use a key or address from a vendor account. Its Presidio example sets presidio_language and pii_entities_config and carries no key, and the page does not say what else Presidio needs to run. The sidebar also lists a built-in LiteLLM content filter. None of these is run here. They need accounts, services or packages this course does not set up, and the page for each guardrail is where its settings are documented.
pre_call vs post_call
| pre_call | post_call | during_call | logging_only | |
|---|---|---|---|---|
| Runs | Before the model call | After the model answers | In parallel with the call | On what is logged |
| Sees | The input | The input and the output | The input | Input and output |
| Can stop the call | Yes, before it starts | Yes, before the reply is returned | Yes, the reply waits for it | No, the client reply is unchanged |
Where guardrails fit
- A rule for every application. A company policy, such as no passwords in tickets, is written once at the gateway.
- Cleaning input. A guardrail that returns changed data can mask a value before a provider sees it.
- Checking answers. A post-call guardrail can stop an answer that breaks a policy before it reaches the user.
default_on: true, and test with a request that does not mention the guardrail.Related
- Previous: The admin UI: seeing what the gateway did
- Next: Observability: watching every call
- Reference: Guardrails quick start and Custom guardrail in the LiteLLM docs
- Change
default_ontofalseand send the password request again. It is no longer refused by the guardrail, and the model answers. - Add
"guardrails": ["no-passwords"]to the request body withdefault_onfalse and send it again. - Change the word the guardrail looks for, restart the gateway and send a request with the new word.
This is what real progress feels like.