How a model writes

Controlling the output

Prompts

In production

Learn how large language models work by calling one: tokens and tokenizers, next-token prediction, streaming, temperature, top_p, length caps and reasoning tokens, seeds, chat messages, system prompts, few-shot examples, structured output, evaluation, context windows, cost and latency. Every reply is a real run of gpt-oss-120b on a free Groq key.

LLM Fundamentals

LLM fundamentals tutorial: how large language models work, in Python

LLM Fundamentals overview

Install groq and tiktoken and set a free Groq API key

Installation and setup

Choosing an LLM: model ids, parameters and hosted vs open weights

Choosing a model

LLM tokens and tokenizers: counting tokens with tiktoken

Tokens

Next-token prediction: how an LLM picks the next word

Next-token prediction

How LLMs generate text one token at a time, and streaming

Generating text

LLM temperature explained with examples in Python

Temperature

LLM top_p and top_k sampling explained

top_p and top_k

LLM max tokens, finish_reason and reasoning tokens

Max tokens and finish_reason

LLM seeds: why the same prompt gives different answers

Seeds

LLM chat messages, roles and chat templates; why calls are stateless

Chat messages and templates

Write a system prompt for an LLM: from vague to structured

System prompts

Few-shot prompting with examples in the messages list

Few-shot prompting

Validate LLM JSON output with Pydantic

Structured output

Evaluate LLM prompts against labelled test cases

Evaluating prompts

LLM context window explained: tokens, history and trimming

Context window

Estimate LLM API cost from input and output tokens

Cost

LLM latency: time to first token and tokens per second

Latency

LLM ticket sorter project: accuracy, cost and latency

Project: ticket sorter

In production quiz

FinCoach's requests grew from 102 to 164 to 218 tokens. Why?