LiteLLMLiteLLM 1.105 · Python 3.10+
Dashboard
0%

Guardrails on every request

A guardrail is a check that the gateway runs on a request or a response, so every application behind the gateway gets the same rule without adding code.

Last updated: 12 Oct, 2026 · LiteLLM 1.105

The admin UI: seeing what the gateway did and the lessons before it set up and watched the gateway. A guardrail changes what it does. Because every call already passes through the gateway, a rule written once there applies to every application, including the ones whose authors never heard of the rule. This lesson writes one guardrail of our own and runs it, and then lists the ready-made ones from the documentation without running them.

Syntax: a guardrail is declared under guardrails in the config, with a name, the guardrail to use and the mode, which says when it runs. The guardrails quick start gives this shape.

yaml
guardrails:
  - guardrail_name: "my-guard"
    litellm_params:
      guardrail: <provider-or-your.module.Class>
      mode: "pre_call"      # when it runs
      default_on: true      # run it for every request

Choosing when a guardrail runs

The quick start page lists four modes. The table later in this lesson compares them. pre_call runs before the model is called and checks the input. post_call runs after the model answers and sees the input and the output. during_call runs in parallel with the model call, and the response is not returned until the check completes. logging_only scans the logged input and output without changing what the client receives. A list such as [pre_call, post_call] runs several modes.

Writing a guardrail of your own

The class and its hook

A guardrail of your own is a class that extends CustomGuardrail from litellm.integrations.custom_guardrail. Its async_pre_call_hook receives the key's details, the cache, the request body and the call type. The custom guardrail page shows this hook and says returning a changed data rewrites the request. Returning data unchanged lets the request through. The page does not show an HTTPException. Raising one to refuse the request with a status of your choice is what the run below demonstrates.

python
from fastapi import HTTPException
from litellm.integrations.custom_guardrail import CustomGuardrail

class NoPasswords(CustomGuardrail):
    """Refuses any request that mentions a password."""

    async def async_pre_call_hook(self, user_api_key_dict, cache, data, call_type):
        for message in data.get("messages", []):
            if "password" in str(message.get("content", "")).lower():
                raise HTTPException(
                    status_code=400,
                    detail={"error": "The desk never handles passwords. Use the reset link."},
                )
        return data

Switching it on in the config

guardrail: desk_guardrail.NoPasswords names the file and the class. mode: pre_call runs it before the model. default_on: true applies it to every request. The quick start page says a default-on guardrail runs even when the request names other guardrails, or none. A guardrail without default_on runs only when the request body lists it in a guardrails array.

Running a request through the guardrail

The example writes the guardrail file and a config, starts a real gateway, and sends two requests. The first is harmless and goes to the Groq model. The second mentions a password and should be refused before any model is called.

ExampleAPI keyA guardrail that refuses before the model is called
import subprocess
import time
import urllib.request
from pathlib import Path

import httpx


def start(port=4400, env=None):
    proc = subprocess.Popen(
        ["litellm", "--config", "config.yaml", "--port", str(port)],
        stdout=open("gateway.log", "w"), stderr=subprocess.STDOUT, env=env,
    )
    while True:
        try:
            urllib.request.urlopen(f"http://127.0.0.1:{port}/health/liveliness")
            return proc
        except OSError:
            time.sleep(1)


Path("desk_guardrail.py").write_text(r'''from fastapi import HTTPException
from litellm.integrations.custom_guardrail import CustomGuardrail


class NoPasswords(CustomGuardrail):
    """Refuses any request that mentions a password."""

    async def async_pre_call_hook(self, user_api_key_dict, cache, data, call_type):
        for message in data.get("messages", []):
            if "password" in str(message.get("content", "")).lower():
                raise HTTPException(
                    status_code=400,
                    detail={"error": "The desk never handles passwords. Use the reset link."},
                )
        return data
''')
Path("config.yaml").write_text("""model_list:
  - model_name: careful
    litellm_params:
      model: groq/openai/gpt-oss-120b
      api_key: os.environ/GROQ_API_KEY

guardrails:
  - guardrail_name: no-passwords
    litellm_params:
      guardrail: desk_guardrail.NoPasswords
      mode: pre_call
      default_on: true

general_settings:
  master_key: sk-gateway-1234
""")

def ask(text):
    r = httpx.post(
        "http://127.0.0.1:4400/chat/completions",
        headers={"Authorization": "Bearer sk-gateway-1234"},
        json={"model": "careful", "max_tokens": 200,
              "messages": [{"role": "user", "content": text}]},
        timeout=60,
    )
    body = r.json()
    if r.status_code != 200:
        return r.status_code, body["error"]["message"]
    return r.status_code, body["choices"][0]["message"]["content"]


proc = start()
try:
    print(*ask("Reply with one word: ok"))
    print(*ask("what is my password"))
finally:
    proc.terminate()
    proc.wait()
  • The first request was answered. It passed the guardrail and the gateway called the Groq model, so its reply is a live model answer and can differ on your run.
  • The second request was refused with 400. The message is the text the guardrail raised, carried back to the caller.
  • No model saw the second request. The guardrail ran in pre_call mode, so the refusal came before the provider was called, and no tokens were spent on it.

The ready-made guardrails

The quick start page shows config for several guardrails supplied by other companies and for Presidio. Its examples for Aporia, Cato Networks, Pillar Security and Azure text moderation each use a key or address from a vendor account. Its Presidio example sets presidio_language and pii_entities_config and carries no key, and the page does not say what else Presidio needs to run. The sidebar also lists a built-in LiteLLM content filter. None of these is run here. They need accounts, services or packages this course does not set up, and the page for each guardrail is where its settings are documented.

pre_call vs post_call

pre_callpost_callduring_calllogging_only
RunsBefore the model callAfter the model answersIn parallel with the callOn what is logged
SeesThe inputThe input and the outputThe inputInput and output
Can stop the callYes, before it startsYes, before the reply is returnedYes, the reply waits for itNo, the client reply is unchanged

Where guardrails fit

  • A rule for every application. A company policy, such as no passwords in tickets, is written once at the gateway.
  • Cleaning input. A guardrail that returns changed data can mask a value before a provider sees it.
  • Checking answers. A post-call guardrail can stop an answer that breaks a policy before it reaches the user.
Watch out. A guardrail that is not default-on protects only the requests that name it. If the rule is a policy, set default_on: true, and test with a request that does not mention the guardrail.
Try it yourself
  • Change default_on to false and send the password request again. It is no longer refused by the guardrail, and the model answers.
  • Add "guardrails": ["no-passwords"] to the request body with default_on false and send it again.
  • Change the word the guardrail looks for, restart the gateway and send a request with the new word.

This is what real progress feels like.