LiteLLMLiteLLM 1.105 · Python 3.10+
Dashboard
0%
1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
48 small wins to finish your pathNext lesson →

Tag routing

Tag routing is a Router feature that picks a deployment by matching the tags on a request with the tags on each deployment.

Last updated: 12 Oct, 2026 · LiteLLM 1.105

Model access groups decide which models a key may call. Tags decide which deployment of a model answers. A deployment is one entry in model_list, and several entries can share a model_name, as Router: model groups and deployments showed. Without tags, Routing strategies: choosing which deployment answers chooses between them. With tags, the caller can steer the choice: a free tier to a cheap deployment, a paid tier to a strong one.

A request for the model chat carries the tag strong. The Router compares it with the tags on the two deployments of chat. The deployment tagged cheap and default is skipped. The deployment tagged strong answers.

Syntax: tags go on each deployment in litellm_params. The setting enable_tag_filtering under router_settings turns the matching on.

yaml
model_list:
  - model_name: chat
    litellm_params:
      model: groq/openai/gpt-oss-20b
      tags: ["cheap", "default"]   # tags on this deployment
router_settings:
  enable_tag_filtering: true      # match request tags to deployment tags

The default tag

The tag routing page says a deployment tagged default receives all untagged requests. The library's own note on the feature adds that when no deployment has the default tag, an untagged request can go to all deployments.

Three ways to send tags

The request tags page lists them: a tags array in the request body, the same array inside metadata, or a header named x-litellm-tags holding a comma-separated list.

text
{"model": "chat", "tags": ["strong"], "messages": [...]}
{"model": "chat", "metadata": {"tags": ["strong"]}, "messages": [...]}
x-litellm-tags: strong,my-custom-tag

Writing the helper that starts a gateway

ops.py starts the litellm command, waits until it answers, and stops it in a finally block. call() sends a request and returns the status, body and headers. chat() sends one chat request, and extra keyword arguments such as tags become fields of the request body.

ExampleWriting ops.py
from pathlib import Path
import sys

Path("ops.py").write_text('''import json
import os
import subprocess
import sys
import time
import urllib.error
import urllib.request
from contextlib import contextmanager

KEY = "sk-gateway-1234"          # the master key written in every config.yaml


@contextmanager
def running(port, env=None):
    """Start the gateway on `port` from config.yaml, wait until it answers, stop it afterwards."""
    litellm = os.path.join(os.path.dirname(sys.executable), "litellm")
    cmd = [litellm, "--config", "config.yaml", "--port", str(port)]
    proc = subprocess.Popen(cmd, env=env, stdout=open("gateway.log", "w"), stderr=subprocess.STDOUT)
    try:
        for _ in range(90):
            if proc.poll() is not None:
                raise RuntimeError("the gateway stopped; read gateway.log")
            try:
                urllib.request.urlopen(f"http://127.0.0.1:{port}/health/liveliness")
                break
            except OSError:
                time.sleep(1)
        yield f"http://127.0.0.1:{port}"
    finally:
        proc.terminate()
        proc.wait(10)


def call(url, path, body=None, key=KEY, headers=None):
    """Send one request. Returns (status, text, headers) and does not raise on 4xx or 5xx."""
    head = dict(headers or {})
    if key:
        head["Authorization"] = f"Bearer {key}"
    if body is not None:
        head["Content-Type"] = "application/json"
    data = json.dumps(body).encode() if body is not None else None
    try:
        reply = urllib.request.urlopen(urllib.request.Request(url + path, data=data, headers=head), timeout=60)
    except urllib.error.HTTPError as error:
        reply = error
    return reply.getcode(), reply.read().decode(), {k.lower(): v for k, v in reply.headers.items()}


def chat(url, model, text="Reply with one word: ok", key=KEY, headers=None, **extra):
    """One chat request. Extra keyword arguments are added to the JSON body."""
    body = {"model": model, "messages": [{"role": "user", "content": text}], "max_tokens": 60, **extra}
    return call(url, "/chat/completions", body, key, headers)
''')
sys.path.insert(0, ".")        # lets the programs below run: from ops import running, call, chat
print("wrote ops.py")

Choosing a deployment with a tag

The config has one group, chat, with two deployments. The 20b model is tagged cheap and default. The 120b model is tagged strong. The program sends five requests and prints the status and the model that answered, read from the x-litellm-model-name response header.

ExampleAPI keyFive requests with different tags
from pathlib import Path
from ops import running, chat

Path("config.yaml").write_text("""model_list:
  - model_name: chat
    litellm_params:
      model: groq/openai/gpt-oss-20b
      api_key: os.environ/GROQ_API_KEY
      tags: ["cheap", "default"]
  - model_name: chat
    litellm_params:
      model: groq/openai/gpt-oss-120b
      api_key: os.environ/GROQ_API_KEY
      tags: ["strong"]
router_settings:
  enable_tag_filtering: true
general_settings:
  master_key: sk-gateway-1234
""")

CASES = [
    ("no tags", {}, {}),
    ("tags: cheap", {"tags": ["cheap"]}, {}),
    ("tags: strong", {"tags": ["strong"]}, {}),
    ("header: strong", {}, {"x-litellm-tags": "strong"}),
    ("tags: unknown", {"tags": ["nonexistent"]}, {}),
]
with running(4741) as url:
    for label, extra, headers in CASES:
        status, _, reply = chat(url, "chat", headers=headers, **extra)
        print(f"{label:15} {status}  {reply['x-litellm-model-name']}")

Reading which deployment answered

  • No tags went to the cheap deployment. It carries default, so it takes untagged requests, as the page says.
  • The tag strong reached the 120b model whether it arrived in the body or in the header.
  • An unknown tag also went to the default deployment. That is what 1.105 did here. The page does not describe this case, so check it on your own version before relying on it.

Excluding and requiring tags

The tag routing page describes two prefixes. A tag that starts with ! excludes deployments that carry that exact tag. A tag that starts with & requires it. The page's table says a request with only ! tags starts from the default-tagged deployments, or from all deployments when none is tagged default, and then removes the excluded ones. If nothing is left, the request fails.

In this config the default-tagged deployment is the cheap one. Excluding cheap on its own therefore removes the whole starting pool. The program sends that request, then two that work.

ExampleAPI keyExcluding the default deployment, then fixing the request
import json
from ops import running, chat

CASES = [
    ("!cheap alone", ["!cheap"]),
    ("strong and !cheap", ["strong", "!cheap"]),
    ("&strong", ["&strong"]),
]
with running(4742) as url:
    for label, tags in CASES:
        status, text, reply = chat(url, "chat", metadata={"tags": tags})
        shown = reply["x-litellm-model-name"] if status == 200 else json.loads(text)["error"]["message"][:100]
        print(f"{label:18} {status}  {shown}")

Reading the exclusion results

  • A request that excludes cheap alone failed with 401. The message is the gateway's own: not allowed to access the model due to the tags configuration. The starting pool was the default-tagged deployment, and the request removed it.
  • Adding a positive tag fixed it. strong together with !cheap chose the 120b deployment.
  • A required tag worked on its own. &strong selected the deployments that carry strong.

Tags on keys, teams and spend

The request tags page also covers tags set on a key or a team in their metadata, and tags that appear in the request_tags field of the spend logs. Its requirements say virtual keys and a database should be set up. None of it is run here. The tag routing page adds allow_fail_open in model_info, which falls back to the default-tagged pool instead of failing, and warns against using it where the constraint is a hard rule.

Tag routing vs routing strategies vs access groups

Tag routingRouting strategyModel access group
Who decidesThe caller, with tagsThe Router, by its own ruleThe key or team settings
DecidesWhich deployment of one nameWhich deployment of one nameWhich names a key may call
Set inDeployment tags and request tagsrouting_strategymodel_info.access_groups

Where you use tag routing

  • Free and paid tiers. A request from a free account carries free and reaches the cheaper deployment.
  • Regions or accounts. Tags such as provider:openai keep a request on an approved provider.
  • Keeping a provider out. A ! tag removes a provider for one request without editing the config.
Watch out. A request can only exclude what the default pool contains. Excluding the deployment that carries default leaves nothing, and the request fails with a message about tags. Add a positive tag, or give the default tag to a deployment you do not exclude.
Try it yourself
  • Remove "default" from the cheap deployment's tags and send a request with no tags. The Router now chooses from all deployments, so the answer changes between calls.
  • Send the tags as metadata={"tags": ["strong"]} instead of tags=["strong"], and read the model that answers.
  • Send the tags ["!cheap", "!strong"] and read the error.
PreviousModel access groups

Every expert started right here.