The MyHDL project contains an extensive test suite. The tests are critical to the quality and expansion of the project. As a developer (new or not) one of the first things you will do is write a test. This document is an introduction to the MyHDL test suite.
MyHDL test suite
The test suite contains two main sets of tests:
Core: The core tests all the hardware description types and extensions in the package. In the repository core tests can be found under: myhdl/test/core
Conversion: The conversion tests verify the conversion from MyHDL to Verilog and VHDL. In the repository conversion tests can be found under: myhdl/test/conversion
The conversion tests are most often added under myhdl/test/conversion/general and make use of convertible testbenches There are additional conversion tests that use cosimulation but these tests are not covered in this document.
The test suite uses pytest as a test framework and test runner. The MyHDL development follows a test-driven-design (TDD) methodology, in most cases this involves having a test generated before a feature is added or a bug is fixed.
The pytest package will need to be installed to run the tests.
As bugs or issues are discovered tests are created to reproduce the issue. An issue description is first entered into the Issue Tracker. Then a test can be created in myhdl/test/bugs/ The test name should contain the github issue number:
test_issue_<github issue #>.py
The tests in the bugs directory are a mix of core and conversion tests depending on the issue. The tests in this directory need to be invoked similar to the tests in the conversion/general (see below).
In addition to pytest an HDL simulator will be required to run the conversion tests. The following is a list of simulators commonly used with the conversion tests. A simulator or analyzer will need to be installed to run the conversion tests.
Icarus Verilog (iverilog): Icarus Verilog is a Verilog simulation and synthesis tool. Icarus is an open-source project
GHDL (ghdl): GHDL is an open source VHDL simulator. An installation guide is available.
Other simulators are supported and it is not difficult to add a new simulator or analyzer.
Running core tests
>> python setup.py develop >> cd myhdl/test/core >> py.test # or make
The above command will run all the tests present in core folder. If you only want to run a specific test you can do the following.
>> py.test <name_of_test>
Running conversion tests
>> cd myhdl/test/conversion/general >> py.test --sim=('iverilog', 'ghdl', 'vlog', 'vcom')
--sim command line argument is used to select the simulator
used in the tests. Many of the tests in the test/bugs
also verify converted results. The bug/issue tests will need the
Keep it Simple: tests do not have to be complex hardware designs. They should be simple enough to quickly understand.
Focus on a specific feature/function: a test should target a specific feature or issue. This reduces complicated tests that can be difficult to understand and maintain.
Fully tested: for the feature/issue being tested make sure all aspects are tested. Test expected failures as well as expected success.
Debug friendly: Try and provide as much information as possible when a test detects an error. This will assist in future debugging.
Files with names
test_*.pyand functions with names
test_*will be executed by the
Functions with names
test_*will be executed as tests by the test runner.
assertstatements in the functions to validate expected results and responses.
The following is a toy example to test the
import pytest import myhdl from myhdl import intbv def test_intbv_assign(): x = intbv(0, min=-8, max=8) x[:] = 4 assert x == 4 with pytest.raises(ValueError): x[:] = -9
Writing conversion tests
The MyHDL package has simulators that are registered in the code
base. As mentioned above a registered simulator is selected by
--sim=<sim> argument with
py.test. Two functions
can be used with the registered simulators to verify converted code.
The analyze and verify functions.
The following example has a valid module and we can check the
conversion using the
analyze function. As mentioned with will
invoke the simulator’s analysis.
def my_valid_module(a, b): @always_comb def digital_logic(): b.next = a return rtl def test_valid_analyze(): a, b = [Signal(bool(0) for _ in range(2)] assert analyze(my_module_exposes_issue, a, b) == 0
The above can be run with:
>> py.test —sim=iverilog test_valid_module.py
The verify function will use a convertible test bench and run the converted code with the Verilog or VHDL simulator. First a convertible test bench is created.
def tb_valid_module(): a, b = [Signal(bool(0)) for _ in range(2)] tbdut = my_valid_module(a, b) @instance def tbstim(): yield delay(2) assert a == b == False a.next = True yield delay(2) assert a == b == True return tbdut, tbstim def test_valid_verify(): assert verify(tb_valid_module) == 0
If the above is added to the same file, we can invoke py.test test same as above and the second test will be run as well.
A template that can be used with Eclipse and othe editors can be found here