Installation and setup

DeepEval installs from PyPI. Its metrics grade with a judge model, OpenAI's by default, and a free Gemini or Groq key can do the judging instead.

pip install deepeval 4.2.3

The version is pinned so your output matches the lessons. DeepEval sends usage telemetry unless DEEPEVAL_TELEMETRY_OPT_OUT=1 is set, and deepeval login connects the Confident AI platform, which this course does not use.

pip install "deepeval==4.2.3"

Example

from importlib.metadata import version

print(version("deepeval"))

A judge model for lesson 22Optional

Lessons up to 21 grade with PretendJudge, a stand-in you write in lesson 9. Lesson 22 hands grading to a hosted model. Every metric takes that model as model=, so the paid OpenAI default can be swapped for a free one.

python

from deepeval.metrics import AnswerRelevancyMetric
from deepeval.models import GeminiModel

judge = GeminiModel(model="gemini-2.5-flash")
metric = AnswerRelevancyMetric(model=judge)

Provider	Key	Variable	DeepEval model
Gemini	Free, aistudio.google.com/apikey	`GOOGLE_API_KEY`	`GeminiModel(model="gemini-2.5-flash")`, needs `google-genai`
Groq	Free, console.groq.com/keys	`GROQ_API_KEY`	`LiteLLMModel(model="groq/openai/gpt-oss-120b")`, needs `litellm`
OpenRouter	Free models end in `:free`, openrouter.ai/keys	`OPENROUTER_API_KEY`	`OpenRouterModel(model="google/gemma-4-31b-it:free")`
OpenAI	Paid, platform.openai.com/api-keys	`OPENAI_API_KEY`	The default, nothing to pass

pip install google-genai

export GOOGLE_API_KEY=AIza...

Try it yourself

Run deepeval --help and find the test and set-gemini commands.

PreviousDeepEval overview

Little by little, you're building something great.