pytest.raises and parametrize: testing the failures

The tests in lesson 30 check answers that are right. The code most likely to fail on the job handles answers that are wrong, so it needs tests too.

The checking step from lesson 25, as a function in its own file:

from typing import Literal

from pydantic import BaseModel, Field


class Triage(BaseModel):
    category: Literal["billing", "shipping", "other"]
    priority: int = Field(ge=1, le=5)


def parse_answer(answer: str) -> Triage:
    return Triage.model_validate_json(answer)

Expecting an error

import pytest
from pydantic import ValidationError

from parse import parse_answer


def test_good_answer():
    result = parse_answer('{"category": "billing", "priority": 4}')
    assert result.category == "billing"


def test_sentence_is_rejected():
    with pytest.raises(ValidationError):
        parse_answer("I am not sure how to sort this one.")

with pytest.raises(ValidationError): passes only if the code inside raises that error. If parse_answer ever starts accepting sentences, this test fails and says so.

pytest -q

One test, many inputs

There are many ways for JSON to be wrong: a category that does not exist, a priority out of range, a field missing. Writing a test function for each repeats the same three lines.

@pytest.mark.parametrize("answer", [
    '{"category": "sales", "priority": 4}',
    '{"category": "billing", "priority": 9}',
    '{"category": "billing"}',
])
def test_bad_values_are_rejected(answer):
    with pytest.raises(ValidationError):
        parse_answer(answer)

@pytest.mark.parametrize is a decorator, like @dataclass in lesson 21. It runs the test once for each value in the list, passing it in as answer.

pytest -q

Five tests from two functions and one list. Adding a new kind of bad answer, when you meet one in real use, is one line in that list.

pytest --collect-only -q

--collect-only lists the tests without running them. Each parametrized run is named after the function plus the input in square brackets, so a failure tells you exactly which answer broke it.