Loading a model with transformers

A language model is two things you download: a tokenizer that turns text into numbers, and weights, the numbers it learned. transformers loads both by name.

from transformers import AutoModelForCausalLM, AutoTokenizer

name = "Qwen/Qwen2.5-0.5B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(name)

print(type(tokenizer).__name__)
print(type(model).__name__)

from_pretrained downloads the files from Hugging Face the first time and reads them from a cache on your disk after that, so only the first run waits for the download.

Auto in the class names means transformers reads the model's configuration and picks the right class for you: here a tokenizer and a model made for Qwen2.

How big is it

parameters = sum(p.numel() for p in model.parameters())
print(f"{parameters:,} parameters")
print(model.config.num_hidden_layers, "layers")
print(model.config.vocab_size, "tokens in the vocabulary")

Parameters are the learned numbers. Half a billion sounds large, and it is small: hosted models are thought to have hundreds of billions. More parameters usually means better answers and more memory, time and money per answer.

numel counts the numbers in one block of weights, and the sum adds up every block. The :, in the f-string puts commas in a long number.