Model access groups

A model access group is a name, set on one or more models in the gateway config, that a key or a team can be granted instead of a list of model names.

Last updated: 12 Oct, 2026 · LiteLLM 1.105

Virtual keys: a key per application gave each application its own key. A key can also be limited to certain models. Writing model names on every key gets hard to keep right: when a model is added, every key that should reach it must be edited. The model access groups page describes the fix. Put a group name on the models once, grant the group to keys, and the page says you can add or remove models from a group without updating individual keys.

Syntax: the group is named in the model's model_info. The key is created with the group name where model names would go.

model_list:
  - model_name: fast
    litellm_params:
      model: groq/openai/gpt-oss-20b
      api_key: os.environ/GROQ_API_KEY
    model_info:
      access_groups: ["beta-models"]   # this model belongs to the group

Granting the group to a key

On the page, a key is created by a request to /key/generate with the group name in models. A key is a row in a database, and this machine has none, as Virtual keys: a key per application showed when /key/generate failed. So this request is not run here.

curl 'http://localhost:4000/key/generate' \
  -H 'Authorization: Bearer <your-master-key>' \
  -H 'Content-Type: application/json' \
  -d '{"models": ["beta-models"]}'

Standing in for the key table with a custom auth function

The custom auth page describes another way to give the gateway a key's details: a Python function receives the key text and returns a UserAPIKeyAuth object, whose models field is the key's model list. The function plays the part of the key table. Everything after it is the gateway's own code: turning a group name into model names, listing the models, and refusing the others.

The page's table says custom auth enforces only rate limits by default. Enforcing the key's models list needs two settings together: custom_auth_run_common_checks under general_settings and enable_post_custom_auth_checks under litellm_settings. The program below runs three gateways to show what each setting changes.

The key and its group

custom_auth.py knows one key, sk-beta-1, and gives it the single entry beta-models. An empty list would mean no restriction, as the page says.

Two models, one group

fast is in the group. smart starts outside it. In the third scenario smart joins the group, and nothing about the key changes.

Writing the helper that starts a gateway

ops.py starts the litellm command, waits until it answers, and stops it in a finally block. call() sends a request and returns the status, body and headers. chat() sends one chat request.

from pathlib import Path
import sys

Path("ops.py").write_text('''import json
import os
import subprocess
import sys
import time
import urllib.error
import urllib.request
from contextlib import contextmanager

KEY = "sk-gateway-1234"          # the master key written in every config.yaml


@contextmanager
def running(port, env=None):
    """Start the gateway on `port` from config.yaml, wait until it answers, stop it afterwards."""
    litellm = os.path.join(os.path.dirname(sys.executable), "litellm")
    cmd = [litellm, "--config", "config.yaml", "--port", str(port)]
    proc = subprocess.Popen(cmd, env=env, stdout=open("gateway.log", "w"), stderr=subprocess.STDOUT)
    try:
        for _ in range(90):
            if proc.poll() is not None:
                raise RuntimeError("the gateway stopped; read gateway.log")
            try:
                urllib.request.urlopen(f"http://127.0.0.1:{port}/health/liveliness")
                break
            except OSError:
                time.sleep(1)
        yield f"http://127.0.0.1:{port}"
    finally:
        proc.terminate()
        proc.wait(10)


def call(url, path, body=None, key=KEY, headers=None):
    """Send one request. Returns (status, text, headers) and does not raise on 4xx or 5xx."""
    head = dict(headers or {})
    if key:
        head["Authorization"] = f"Bearer {key}"
    if body is not None:
        head["Content-Type"] = "application/json"
    data = json.dumps(body).encode() if body is not None else None
    try:
        reply = urllib.request.urlopen(urllib.request.Request(url + path, data=data, headers=head), timeout=60)
    except urllib.error.HTTPError as error:
        reply = error
    return reply.getcode(), reply.read().decode(), {k.lower(): v for k, v in reply.headers.items()}


def chat(url, model, text="Reply with one word: ok", key=KEY, headers=None, **extra):
    """One chat request. Extra keyword arguments are added to the JSON body."""
    body = {"model": model, "messages": [{"role": "user", "content": text}], "max_tokens": 60, **extra}
    return call(url, "/chat/completions", body, key, headers)
''')
sys.path.insert(0, ".")        # lets the programs below run: from ops import running, call, chat
print("wrote ops.py")

wrote ops.py

Calling two models with a key limited to a group

For each scenario the program lists the models the key can see, then asks fast and smart for one short answer. When a call fails it prints the error type from the response body.

import json
from pathlib import Path
from ops import running, call, chat

Path("custom_auth.py").write_text("""from fastapi import Request
from litellm.proxy._types import UserAPIKeyAuth

KEYS = {"sk-beta-1": ["beta-models"]}      # key text -> the models it may call


async def user_api_key_auth(request: Request, api_key: str) -> UserAPIKeyAuth:
    if api_key not in KEYS:
        raise Exception("unknown key")
    return UserAPIKeyAuth(api_key=api_key, models=KEYS[api_key])
""")

TEMPLATE = """model_list:
  - model_name: fast
    litellm_params:
      model: groq/openai/gpt-oss-20b
      api_key: os.environ/GROQ_API_KEY
    model_info:
      access_groups: ["beta-models"]
  - model_name: smart
    litellm_params:
      model: groq/openai/gpt-oss-120b
      api_key: os.environ/GROQ_API_KEY
    model_info:
      access_groups: {smart_groups}
general_settings:
  master_key: sk-gateway-1234
  custom_auth: custom_auth.user_api_key_auth
  custom_auth_run_common_checks: {flags}
litellm_settings:
  enable_post_custom_auth_checks: {flags}
"""
SCENARIOS = [
    ("checks off", False, []),
    ("checks on", True, []),
    ("checks on, smart joins the group", True, ["beta-models"]),
]
for number, (label, flags, smart_groups) in enumerate(SCENARIOS):
    Path("config.yaml").write_text(TEMPLATE.format(flags=json.dumps(flags), smart_groups=json.dumps(smart_groups)))
    with running(4731 + number) as url:
        listed = json.loads(call(url, "/v1/models", key="sk-beta-1")[1])["data"]
        print(label, "| listed:", [m["id"] for m in listed])
        for model in ["fast", "smart"]:
            status, text, _ = chat(url, model, key="sk-beta-1")
            print("   ", model, status, "" if status == 200 else json.loads(text)["error"]["type"])

checks off | listed: ['fast']
    fast 200 
    smart 200 
checks on | listed: ['fast']
    fast 200 
    smart 403 key_model_access_denied
checks on, smart joins the group | listed: ['fast', 'smart']
    fast 200 
    smart 200 

Reading the three scenarios

• With the checks off, the group was ignored on the call. The model list already showed only fast, yet smart answered 200. A key limit that the gateway does not enforce looks like a working limit until someone tests it.
• With the checks on, smart was refused. The gateway answered 403 with the error type key_model_access_denied. The docs page shows a different message, Invalid model for key, for a key made with /key/generate. That path is not run here.
• When smart joined the group, the same key reached it. The key text did not change. One edit to the config moved a model into the group. Here the gateway had to be restarted because the models come from the config file.
• The model list follows the group. /v1/models returned only the models the key may call, which is how a client can discover its own access.

Managing groups without a restart

The page describes API endpoints that create, read and update a group while the gateway runs: /access_group/new, /access_group/<name>/info and /access_group/<name>/update. It says they work only for models stored in the database through /model/new, not for models written in config.yaml. It also says teams can be granted a group with /team/new, and that a group can carry one shared budget. None of this is run here, because each needs a database.

The page marks access groups on wildcard models, such as openai/*, as an Enterprise feature.

Access groups vs model names on a key

Where you use access groups

• Production and beta models. Keys for the live product get the stable group. Test keys get the beta group.
• Expensive models. A strong model sits in a group that only some teams receive.
• Provider families. A group per provider, so a new model from one provider reaches the right keys on its own.

Related

• Previous: Alerting
• Next: Tag routing
• See also: Gateway keys for the master key and how the gateway checks keys
• Reference: Model Access Groups and Custom Auth in the LiteLLM docs