# Improve Python testing with parameterisation

Parameterisation is a technique which makes testing simpler, more concise and more effective. It does this by separating test logic from test data. Let's consider a test for a simple function, and how it can be improved by parameterisation.

## Simple test

Let's test the following function:

```
# prime.py
import math
def is_prime(x):
"""
is_prime returns True or False indicating whether x is prime or not.
"""
if x <= 1:
return False
for i in range(2, int(math.sqrt(x))):
if x % i == 0:
return False
return True
```

We can test this for a range of inputs with the following code:

```
# test_prime.py
import unittest
from prime import is_prime
class TestIsPrime(unittest.TestCase):
def test_x_negative(self):
self.assertEqual(is_prime(-1), False)
def test_x_zero(self):
self.assertEqual(is_prime(0), False)
def test_x_one(self):
self.assertEqual(is_prime(1), False)
def test_x_two(self):
self.assertEqual(is_prime(2), True)
def test_x_three(self):
self.assertEqual(is_prime(3), True)
def test_x_ten(self):
self.assertEqual(is_prime(10), False)
def test_x_fifty_three(self):
self.assertEqual(is_prime(53), True)
if __name__ == "__main__":
unittest.main()
```

We run the tests with:

```
$ python test_prime.py
Ran 7 tests in 0.000s
OK
```

All our tests pass, but our test code is verbose. Although each test is basically the same, we need to add a new method for each one. There is a lot of repeated boilerplate code.

## Parameterisation

We can reduce this boilerplate by parameterising the tests:

```
# test_prime.py
import unittest
from prime import is_prime
class TestIsPrime(unittest.TestCase):
def test_is_prime(self):
test_cases = [
(-1, False),
(0, False),
(1, False),
(2, True),
(3, True),
(10, False),
(53, True),
]
for x, output in test_cases:
self.assertEqual(is_prime(x), output)
if __name__ == "__main__":
unittest.main()
```

We've extracted the test data into the `test_cases`

variable. The test logic is
then run on each of the test cases in turn. This is an improvement, but still
has some flaws. When we run the test, we get the following output:

```
$ python test_prime.py
Ran 1 test in 0.000s
OK
```

The test output says we're running one test, even though we still have seven test cases. Even worse, if one of our tests fails, we aren't given any information on which test failed:

```
$ python test_prime.py
F
======================================================================
FAIL: test_is_prime (__main__.TestIsPrime)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test_prime_parameterised.py", line 19, in test_is_prime
self.assertEqual(is_prime(x), output)
AssertionError: True != False
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (failures=1)
```

## Parameterisation with subTest

Python 3.4 introduced a way to solve this problem. We can use the
`unittest.TestCase.subTest`

context manager for explicit parameterisation:

```
import unittest
from prime import is_prime
class TestIsPrime(unittest.TestCase):
def test_is_prime(self):
test_cases = [
(-1, False),
(0, False),
(1, False),
(2, True),
(3, True),
(10, True),
(53, True),
]
for x, output in test_cases:
with self.subTest(name=str(x)):
self.assertEqual(is_prime(x), output)
if __name__ == "__main__":
unittest.main()
```

If a test fails, `unittest`

will print out the name of the failed test:

```
$ python test_prime.py
======================================================================
FAIL: test_is_prime (__main__.TestIsPrime) (name='10')
----------------------------------------------------------------------
Traceback (most recent call last):
File "test_prime.py", line 20, in test_is_prime
self.assertEqual(is_prime(x), output)
AssertionError: False != True
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (failures=1)
```

The `subTest`

feature was only added in Python 3.4, but has been backported to
Python 2.7 onwards with the unittest2
package.

## Parameterisation with pytest

The pytest framework also solves these
problems for us, and is compatible with Python 2. We can install it with
`pip install pytest`

.

Pytest contains a feature which allows us to parameterise test cases:

```
# test_prime.py
import pytest
from prime import is_prime
@pytest.mark.parametrize("x,output", [
(-1, False),
(0, False),
(1, False),
(2, True),
(3, True),
(10, False),
(53, True),
])
def test_is_prime(x, output):
assert is_prime(x) == output
```

Parameterisation is implemented with the `pytest.mark.parametrize`

decorator.
This decorator takes two arguments. The first is a comma separated string of the
names given to the test cases defined in the second argument. The second
argument is a list of tuples. Each tuple contains the data needed for a test
case.

The decorator makes seven separate calls to `test_is_prime`

, supplying each of
the test cases in turn. When we run the tests, we see that seven tests are run:

```
$ pytest test_prime_pytest.py
test_prime_pytest.py .......
7 passed in 0.02 seconds
```

Importantly, when a test fails, pytest gives us information about which test failed:

```
$ pytest test_prime.py
test_prime_pytest.py ...F...
FAILURES
test_is_prime[2-False]
x = 2, output = False
> assert is_prime(x) == output
E assert True == False
E + where True = is_prime(2)
test_prime_pytest.py:16: AssertionError
1 failed, 6 passed in 0.04 seconds
```

## Conclusion

Parameterising tests is a powerful technique. By separating test logic from data, the focus is shifted away from boilerplate code and onto testing features. It becomes trivial to add new test cases.

Parameterised tests work best with pure functions. A pure function is a function which satisfies the following constraints:

- Its return value is determined exclusively by the input values (e.g.. it doesn't use global or object variables)
- Its execution doesn't cause any side effects (e.g. it doesn't print or write data to a file)

A pure function's behaviour depends only on the arguments passed to it, so they can often be exhaustively tested with a single parameterised test.

**Change log**:

- The first version of this article didn't contain the section on Parameterisation with subTest. Thanks to @rochacbruno and @ossronny for pointing it out.