verbose_mode: seeing how a score was made

Lesson 3 printed a score and a reason. When a score surprises you, the next question is how it was reached, and every metric can print its own working.

python

from deepeval.metrics import ExactMatchMetric
from deepeval.test_case import LLMTestCase

reworded = LLMTestCase(
    input="Where is order A17?",
    actual_output="Your order A17 was shipped on 3 March.",
    expected_output="Order A17 shipped on 3 March.",
)

The reworded answer from lesson 3, and a metric created with one extra argument.

Example

metric = ExactMatchMetric(verbose_mode=True)
metric.measure(reworded)

Nothing was printed by the snippet itself. The metric printed its log while it measured: the score and the reason, between two rules.

For exact match that is the whole story, because there is only one step. A metric that asks a judge has several steps, and verbose mode prints each one: what the judge was asked and what came back. Lesson 9 turns it on for one of those, and the log gets much longer.

The log, kept on the metric

Example

print(repr(metric.verbose_logs))

The same text is kept in verbose_logs, so a script can save it next to a failing result instead of scrolling back to find it.

For a whole run at once

The command that runs evals as tests, deepeval test run, has a -v flag that turns verbose mode on for every metric. It arrives in lesson 20.

Try it yourself

Measure the first, correct answer with verbose mode on and compare the two logs.
Measure two test cases in a row with the same metric and print verbose_logs. Which one does it hold?

PreviousExactMatchMetric: your first metric

This is what real progress feels like.