Marking Python Tests as Optional

Often code tests are to be run on special demand, rather than in a CI/CD manner: for instance, they may be slow or work only in a local mode with protected data. This note shows how to declare code tests optional in pytest, the leading testing framework for Python. The article is inspired by the pytest documentation on test markers.

The trick is to mark extra tests with a decorator and couple it with a runtime flag.
The first step is to mark the relevant code blocks and is illustrated below:

import pytest

@pytest.mark.optional
def test_addition():
    assert 1+1==2

The second step is to instruct the config flag on how to interpret the decorator; it is also worth describing this action in the helper. The code below shows how to set this up:

import pytest

def pytest_addoption(parser):
    """Introduce a flag for optional tests"""
    parser.addoption(
        "--optional", action="store_true", default=False, help="run extra tests"
    )

def pytest_collection_modifyitems(config, items):
    """Instruct the testing framework to obey the decorator for optional tests"""
    if config.getoption("--optional"):
        # --extras given in cli: do not skip extra tests
        return
    # otherwise, execute
    skip_extras = pytest.mark.skip(reason="need --optional flag to run")
    for item in items:
        if "optional" in item.keywords:
            item.add_marker(skip_extras)

def pytest_configure(config):
    """Make the decorator for optional tests visible in help: pytest --markers"""
    config.addinivalue_line("markers", "optional: mark test as optional to run")

To see it in action, run pytest -k battery --optional 😎

test_battery.py::test_addition PASSED

Published by mskorski

Scientist, Consultant, Learning Enthusiast

Leave a comment

Your email address will not be published. Required fields are marked *