Unit and Documentation Testing¶
As described in Showing Code Snippets, Sphinx integrates with
doctest module to help ensure that both the documentation
and underlying library are correct. Doctests consist of short snippets
of code along with their expected output. A doctest passes if the
actual output of the snippet matches its expected output. For instance,
a doctest that
1 + 1 correctly produces
2 could be written as:
Below, we show an example of addition in practice: >>> 1 + 1 2 Later, we will consider more advanced operators.
The blank lines above and below the doctest separate it from the surrounding text, while the expected output appears immediately below the relevant code.
To run the doctests in the QInfer documentation using Linux or OS X:
$ cd doc/ $ make doctest
To run the doctests on Windows using PowerShell, use
PS > cd doc/ PS > .\make doctest
As with the unit tests, doctests are automatically run on pull requests, to help ensure the correctness of contributed documentation.
A doctest snippet may be annotated with one or more comments that change
the behavior of that test. The
doctest documentation goes into far
more detail, but the two we will commonly need are
# doctest: +SKIP
# doctest: +ELLIPSIS. The former causes a test to be skipped entirely.
Skipping tests can be useful if the output of a doctest is random, for instance.
+ELLIPSIS, causes any ellipsis (
...) in the expected output
to act as a wild card. For instance, both of the following doctests would pass:
>>> print([1, 2, 3]) [1, ..., 3] >>> print([1, 2, 3, 4, 'foo', 3]) [1, ..., 3]
There are a few annoyances that come along with writing tests based on string-equivalence of outputs, in particular for a cross-platform and 2/3 compatible library. In particular:
NumPy, 2to3 and
class: Python 2 and 3 differ on whether the type of NumPy
>>> print(type(np.array())) <type 'numpy.ndarray'> # Python 2 <class 'numpy.ndarray'> # Python 3
Thus, to write a doctest that checks if something is a NumPy array or not, it is preferred to use an
>>> isinstance(np.array(), np.ndarray) True
Though this sacrifices some on readability, it gains on portability and correctness.
longin array shapes: The Windows and Linux versions of NumPy behave differently with respect to when a NumPy shape is represented as a
longvalues. This can cause doctests to choke on spurious
>>> print(np.zeros((1, 10)).shape) (1, 10) # tuple of int (1L, 10L) # tuple of long
==, however, the same trick as above can help:
>>> np.zeros((1, 10)).shape == (1, 10) True