Building HDL and Running Tests

New in version 1.8.

Warning

Python runners and associated APIs are an experimental feature and subject to change.

The Python Test Runner (short: “runner”) described here is a replacement for cocotb’s traditional Makefile flow. It builds the HDL for the simulator and runs the tests.

It is not meant to be ideal solution for everyone. You can easily integrate cocotb into your custom flow, see Extending existing build flows.

Usage with pytest

The runner can be used with pytest which is Python’s go-to testing tool.

For an example on how to set up the runner with pytest, see the file cocotb-root/examples/simple_dff/test_dff.py, with the relevant part shown here:

def test_simple_dff_runner():

    hdl_toplevel_lang = os.getenv("HDL_TOPLEVEL_LANG", "verilog")
    sim = os.getenv("SIM", "icarus")

    proj_path = Path(__file__).resolve().parent

    verilog_sources = []
    vhdl_sources = []

    if hdl_toplevel_lang == "verilog":
        verilog_sources = [proj_path / "dff.sv"]
    else:
        vhdl_sources = [proj_path / "dff.vhdl"]

    runner = get_runner(sim)()
    runner.build(
        verilog_sources=verilog_sources, vhdl_sources=vhdl_sources, hdl_toplevel="dff"
    )

    runner.test(hdl_toplevel="dff", test_module="test_dff")

You run this file with pytest like

SIM=questa HDL_TOPLEVEL_LANG=vhdl pytest examples/simple_dff/test_dff.py

Note that the environment variables SIM and HDL_TOPLEVEL_LANG are defined in this test file to set arguments to the runner’s build() and test() functions; they are not directly handled by the runner itself.

Test filenames and functions have to follow the pytest discovery convention in order to be automatically found.

By default, pytest will only show you a terse “pass/fail” information. To see more details of the simulation run, including the output produced by cocotb, add the -s option to the pytest call:

SIM=questa HDL_TOPLEVEL_LANG=vhdl pytest examples/simple_dff/test_dff.py -s

Note

Take a look at the list of command line flags in pytest’s official documentation.

Direct usage

You can also run the test directly, that is, without using pytest, like so

python examples/simple_dff/test_dff.py

This requires that you define the test to run in the testcase file itself. For example, add the following code at the end:

if __name__ == "__main__":
    test_simple_dff_runner()

API

The API of the Python runner is documented in section Python Test Runner.