Building HDL and Running Tests with Python

Building HDL and Running Tests with Python#

Added in version 1.8.

Warning

Python runners and associated APIs are an experimental feature and subject to change.

The Python Test Runner (short: “runner”) described here is a replacement for cocotb’s traditional Makefile flow. It builds the HDL for the simulator and runs the tests.

It is not meant to be the ideal solution for everyone. You can easily integrate cocotb into your custom flow, see Extending Existing Build Flows.

Usage with pytest#

The runner can be used with pytest which is Python’s go-to testing tool.

For an example on how to set up the runner with pytest, see the file cocotb-root/examples/simple_dff/test_dff.py, with the relevant part shown here:

You run this file with pytest like

SIM=questa TOPLEVEL_LANG=vhdl pytest examples/simple_dff/test_dff.py

Note that the environment variables SIM and TOPLEVEL_LANG are defined in this test file to set arguments to the runner’s Runner.build and Runner.test functions; they are not directly handled by the runner itself.

Test filenames and functions have to follow the pytest discovery convention in order to be automatically found.

By default, pytest will only show you a terse “pass/fail” information. To see more details of the simulation run, including the output produced by cocotb, add the -s option to the pytest call:

SIM=questa TOPLEVEL_LANG=vhdl pytest examples/simple_dff/test_dff.py -s

Note

Take a look at the list of command line flags in pytest’s official documentation.

Usage with plugin#

cocotb_tools.pytest.plugin is a plugin for pytest that will allow to use pytest as regression manager for cocotb tests and extend cocotb testing capabilities with pytest features like fixtures.

Please see the chapter about Pytest Support to learn how to enable plugin in your project.

Above example will just run fine with enabled plugin. But we could also slightly refactor it to gain all benefits from using pytest like showing all available cocotb tests in our project with pytest --collect-only or adding support for command line arguments.

import pytest
from cocotb_tools.pytest.hdl import HDL


@pytest.fixture(name="dff")
def dff_fixture(hdl: HDL) -> HDL:
    """Define new HDL design by using pytest fixture."""
    hdl.toplevel = "dff"

    proj_path = Path(__file__).resolve().parent

    if hdl.toplevel_lang == "verilog":
        hdl.sources = (proj_path / "dff.sv",)
    else:
        hdl.sources = (proj_path / "dff.vhdl",)

    hdl.build(always=True)

    return hdl


@pytest.mark.cocotb_runner  # NOTE: mark this test function as cocotb runner
def test_simple_dff_runner(dff: HDL) -> None:
    """Run HDL simulator with cocotb tests to test DFF."""
    dff.test()


# NOTE: When using plugin, there is no need for using @pytest.mark.cocotb_test or @cocotb.test decorators
async def test_simple_dff_feature_1(dut) -> None:
    """Test DFF feature 1."""

To run it:

pytest examples/simple_dff/test_dff.py -s --cocotb-toplevel-lang=vhdl --cocotb-simulator=questa

Direct usage#

You can also run the test directly, that is, without using pytest, like so

python examples/simple_dff/test_dff.py

This requires that you define the test to run in the testcase file itself. For example, add the following code at the end:

if __name__ == "__main__":
    test_simple_dff_runner()

Generate waveforms#

For many simulators it is possible to generate simulation waveforms. This can be done by setting the waves argument to True in the Runner.build and Runner.test functions. It is also possible to set the environment variable WAVES to a non-zero value to generate waveform files at run-time without modifying the test code, e.g.,

WAVES=1 pytest examples/simple_dff/test_dff.py

Similarly, it is possible to disable the waveform generation set in the test code by setting WAVES to 0.

Starting in GUI mode/viewing waveforms#

To start the simulator GUI or, for simulators not having a GUI, view the waveform file in a waveform viewer after the simulation, it is possible to set the gui argument to True in Runner.test. It is also possible to set the GUI environment variable to a non-zero value, e.g.,

GUI=1 pytest examples/simple_dff/test_dff.py

For simulators without a GUI, cocotb will open a waveform viewer. Either Surfer or GTKWave, in that order, if available in the system path. To specify preferred waveform viewer, the COCOTB_WAVEFORM_VIEWER environment variable can be used. This can also be used to set, e.g., the surfer.sh startup script for WSL.

API#

The API of the Python runner is documented in section Python Test Runner.