Installation and setup

This course runs a small open model with PyTorch and transformers. The model downloads from Hugging Face without an account; gated models need a token.

torch, transformers and tiktoken

Allow about 3 GB of free disk space and 4 GB of free memory. torch does the maths, transformers loads and runs models from Hugging Face, and tiktoken is OpenAI's tokenizer, used in lesson 16. The model itself downloads the first time lesson 2 runs.

pip install "torch==2.14.0" "transformers==5.17.0" "tiktoken==0.14.0" pydantic

Example

from importlib.metadata import version

print(version("transformers"))

A Hugging Face token for gated modelsOptional

Qwen2.5-0.5B-Instruct is public, so it downloads without an account. Gated models, whose licence you have to accept on Hugging Face first, need a token: create one in your Hugging Face settings and log in with hf auth login, or set HF_TOKEN. A token also raises download rate limits.

hf auth login

Downloaded models are cached under ~/.cache/huggingface; set HF_HOME to keep them somewhere else. Hosted model APIs, with their own keys, are covered in APIs for AI.

Try it yourself

Run hf auth whoami to see whether you are logged in.

PreviousLLM Fundamentals overview

Little by little, you're building something great.