JUnit Tests Report Reference

JUnit Tests Report Reference#

This document describes the format of the JUnit XML test report files generated by cocotb.

This formats aims to support a wide range of different CI environments like:

  • GitHub Actions (with the help of junit-report-action or similar actions)

  • GitLab CI

  • Jenkins CI

The result XML files are compatible with Jenkins xUnit schema version 2 and with JUnit XML test report files from pytest when used with the junit_family=xunit2 pytest option. However, only common fields are supported.

Attributes#

testsuites#

The root of the XML document. This represents all Python test modules in a regression run. It contains list of testsuite elements.

List of supported attributes:

  • name - name of the regression.

Example:

<testsuites name="cocotb tests">
</testsuites>

testsuite#

A testsuite represents all tests from a single Python module. It contains list of testcase elements.

List of supported attributes:

  • name - name of testsuite based on name of Python module where tests are defined.

  • errors - total number of errors from all tests reported in this testsuite.

  • failures - total number of failed tests reported in this testsuite.

  • skipped - total number of skipped tests reported in this testsuite.

  • tests - total number of tests reported in this testsuite.

  • time - total wall time of all tests in this testsuite in seconds.

  • timestamp - date and time of when the testsuite run was executed (in the ISO 8601 format with the UTC timezone).

  • hostname - name of the machine where the testsuite run was executed.

Note

The Jenkins xUnit schema version 2 doesn’t allow the file and line attributes in testcase elements. They are part of property elements.

Note

When generating a JUnit XML file from pytest, it is possible to include file and line as attributes in testcase elements by setting the pytest junit_family to xunit1:

pytest --override-ini=junit_family=xunit1 --junit-xml=junit.xml ...

Example:

<testsuite name="test_dff" errors="0" failures="1" skipped="1" tests="3" time="0.001" timestamp="2025-12-28T12:52:42.287738+00:00" hostname="d9019dc0cfcf">
</testsuite>

testcase#

A single cocotb test.

It may contain properties, skipped, failure, error, system-out and system-err elements.

List of supported attributes:

  • classname - Python module where the reported test was defined.

  • name - name of the reported test, mostly the name of a Python function or a Python class.

  • time - total wall time of executed test in seconds.

Example:

<testcase classname="test_dff" name="dff_simple_test" time="0.001">
</testcase>

properties#

List of property elements.

No attributes.

<properties>
  <property name="cocotb" value="True" />
</properties>

property#

Additional data (name-value pairs) for a testcase that cannot be represented as XML attribute in the testcase XML element.

List of supported attributes:

  • name - name of test property.

  • value - value of test property.

List of supported properties:

  • cocotb - marks the testcase as a cocotb test.

  • file - path to source file where test was defined.

  • line - line number in file where test was defined.

  • sim_time_unit - simulation time unit like ns (nanoseconds).

  • sim_time_duration - total simulation time of executed test in sim_time_unit.

  • sim_time_start - simulation time when test started in sim_time_unit.

  • sim_time_stop - simulation time when test finished in sim_time_unit.

  • sim_time_ratio - ratio of wall time to simulation time.

  • random_seed - value of random seed set for test.

  • attachment - file attachment like generated simulation log or waveform files.

Example:

<properties>
  <property name="cocotb" value="True" />
  <property name="random_seed" value="1766926362" />
  <property name="sim_time_unit" value="ns" />
  <property name="sim_time_duration" value="115000.0" />
  <property name="sim_time_ratio" value="139607803.1837916" />
  <property name="attachment" value="sim_build/dff.ghw" />
  <property name="file" value="examples/simple_dff/test_dff.py" />
  <property name="line" value="17" />
  <property name="attachment" value="sim_build/dff.vcd" />
  <property name="attachment" value="sim_build/dff.log" />
</properties>

failure#

If present, test failed.

List of supported attributes:

  • message - message of raised exception.

  • type - (optional) type of raised exception like AssertionError.

The text part of failure element will contain traceback of raised exception.

Example:

<failure message="value must be equal 5" type="AssertionError">Traceback (most recent call last):
  File "examples/simple_dff/test_dff.py", line 49, in dff_simple_test_failed
    assert value == 5, "value must be equal to 5"
AssertionError: value must be equal to 5
assert 3 == 5
</failure>

skipped#

If present, test was skipped.

List of supported attributes:

  • message - (optional) reason why test was skipped.

Example:

<skipped message="Test was skipped" />

error#

If present, test finished with an unexpected error.

List of supported attributes:

  • message - message of raised exception or error message.

  • type - (optional) type of raised exception like RuntimeError.

The optional text part of failure element will contain traceback of raised exception.

system-out#

Optional. It may contain captured standard output of test and file attachments.

No attributes.

Example:

<system-out>[[ATTACHMENT|sim_build/dff.ghw]]
[[ATTACHMENT|sim_build/dff.log]]
</system-out>

system-err#

Optional. It may contain captured standard error of test.

No attributes.

Example:

<system-err>Test failed with COCOTB_RANDOM_SEED=1766926362
</system-err>

Examples#

This example shows a regression with 1 testsuite (test_dff) with 3 tests. The first test (dff_simple_test) passed, the next (dff_simple_test_failed) failed, and the last (dff_simple_test_skipped) was skipped.

<?xml version='1.0' encoding='utf-8'?>
<testsuites name="cocotb tests">
  <testsuite name="test_dff" errors="0" failures="1" skipped="1" tests="3" time="0.001" timestamp="2025-12-28T12:52:42.287738+00:00" hostname="d9019dc0cfcf">
    <testcase classname="test_dff" name="dff_simple_test" time="0.001">
      <properties>
        <property name="cocotb" value="True" />
        <property name="random_seed" value="1766926362" />
        <property name="sim_time_unit" value="ns" />
        <property name="sim_time_duration" value="115000.0" />
        <property name="sim_time_ratio" value="139607803.1837916" />
        <property name="attachment" value="sim_build/dff.ghw" />
        <property name="file" value="examples/simple_dff/test_dff.py" />
        <property name="line" value="17" />
      </properties>
      <system-out>[[ATTACHMENT|sim_build/dff.ghw]]
</system-out>
    </testcase>
    <testcase classname="test_dff" name="dff_simple_test_failed" time="0.000">
      <properties>
        <property name="cocotb" value="True" />
        <property name="random_seed" value="1766926362" />
        <property name="sim_time_unit" value="ns" />
        <property name="sim_time_duration" value="0.0" />
        <property name="sim_time_ratio" value="0.0" />
        <property name="attachment" value="sim_build/dff.ghw" />
        <property name="file" value="examples/simple_dff/test_dff.py" />
        <property name="line" value="45" />
      </properties>
      <failure message="value must be equal 5" type="AssertionError">Traceback (most recent call last):
  File "examples/simple_dff/test_dff.py", line 49, in dff_simple_test_failed
    assert value == 5, "value must be equal to 5"
AssertionError: value must be equal to 5
assert 3 == 5
</failure>
      <system-out>[[ATTACHMENT|sim_build/dff.ghw]]
</system-out>
      <system-err>Test failed with COCOTB_RANDOM_SEED=1766926362
</system-err>
    </testcase>
    <testcase classname="test_dff" name="dff_simple_test_skipped" time="0.000">
      <properties>
        <property name="cocotb" value="True" />
        <property name="random_seed" value="1766926362" />
        <property name="sim_time_unit" value="ns" />
        <property name="sim_time_duration" value="0.0" />
        <property name="sim_time_ratio" value="0.0" />
        <property name="attachment" value="sim_build/dff.ghw" />
        <property name="file" value="examples/simple_dff/test_dff.py" />
        <property name="line" value="52" />
      </properties>
      <skipped message="Test was skipped" />
      <system-out>[[ATTACHMENT|sim_build/dff.ghw]]
</system-out>
    </testcase>
  </testsuite>
</testsuites>