EvaluationResult: reading results in code

The report in lesson 5 is for reading. evaluate also returns every result as data, and that is what a script needs when it has to do something about a failure.

from deepeval import evaluate
from deepeval.evaluate import DisplayConfig

result = evaluate(
    test_cases=cases,
    metrics=[ExactMatchMetric()],
    display_config=DisplayConfig(print_results=False, show_indicator=False),
)
for test in result.test_results:
    data = test.metrics_data[0]
    print(test.name, test.success, data.name, data.score, data.reason)

DisplayConfig controls what evaluate prints. print_results=False removed the boxes and show_indicator=False removed the progress line.

The summary still printed. In deepeval 4.2.3, print_results hides the report on each test case and nothing else, so every run in this course ends with those lines.

What came back

result.test_results has one entry per test case, in the order you passed them. Each one is named test_case_0, test_case_1 and so on, and carries its own success.

metrics_data has one entry per metric on that test case, with the metric's name, score, reason and threshold. A test case passes only when every metric on it passes.

failed = [test.name for test in result.test_results if not test.success]
print("failed:", failed)

A list of failures is enough to act on. A script can stop with an error when it is not empty, which is all a build server needs to go red. Lesson 20 shows DeepEval's own way of doing that.