Installation and setup

Haystack's package on PyPI is haystack-ai. Its OpenAI chat generator accepts any OpenAI-compatible URL, which makes free Groq and Gemini keys usable.

pip install haystack-ai 3.1.1

The package name is haystack-ai; the older farm-haystack is Haystack 1 and does not work with this course. Haystack sends anonymous usage statistics to deepset, and the outputs here were captured with HAYSTACK_TELEMETRY_ENABLED=False.

pip install "haystack-ai==3.1.1" pytest

Example

from importlib.metadata import version

print(version("haystack-ai"))

A chat model behind OpenAIChatGeneratorOptional

BM25 retrieval needs no model, and lesson 11 writes a stand-in chat generator, so the course runs without a key. OpenAIChatGenerator reads OPENAI_API_KEY by default. Point it at another provider with api_base_url and that provider's key.

Provider	Key	Variable	api_base_url
Groq	Free, console.groq.com/keys	`GROQ_API_KEY`	`https://api.groq.com/openai/v1`
Gemini	Free, aistudio.google.com/apikey	`GEMINI_API_KEY`	`https://generativelanguage.googleapis.com/v1beta/openai/`
OpenRouter	Free models end in `:free`, openrouter.ai/keys	`OPENROUTER_API_KEY`	`https://openrouter.ai/api/v1`
OpenAI	Paid, platform.openai.com/api-keys	`OPENAI_API_KEY`	Leave it unset

export GROQ_API_KEY=gsk_...

Haystack also ships generators of its own for two of these: google-genai-haystack brings GoogleGenAIChatGenerator and openrouter-haystack brings OpenRouterChatGenerator.

python

from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.utils import Secret

generator = OpenAIChatGenerator(
    api_key=Secret.from_env_var("GROQ_API_KEY"),
    api_base_url="https://api.groq.com/openai/v1",
    model="openai/gpt-oss-120b",
)

Try it yourself

Print OpenAIChatGenerator.__init__.__doc__ and read what api_base_url is for.

PreviousHaystack overview

Little by little, you're building something great.