Programming in Python

Unit Testing - Systematic Search for Errors

Gerald Senarclens de Grancy

Purpose of Unit Tests

What is a Unit Test

  • Allows automated testing of independent code units
  • No replacement for other test categories
  • Write them as early as possible - if possible before the code it tests
  • Tests can serve as useful requirements specification
  • Create a test for each bug you encounter (refacturing!)

Which problems does unit testing solve?

Code without tests (aka. legacy code) is extremely difficult to work with

Unit tests...

  • are a first line of defense against bugs
  • improve code quality by dictating testable code
  • allow releasing higher quality early and often
  • give more confidence to programmers
  • are a prerequisit to refactoring
  • can be used as a contract with regard to desired behavior
  • prevent regressions (bugs in features priorly working properly)

Which problems does unit testing cause?

Essentially none, but writing unit tests ...

  • seems to cost time
  • demands discipline
  • is far from trivial
  • does not substitute other kinds of testing

What is a Test(case)

  • Answers a single question about the code it is testing
  • Run completely by itself (automation)
  • Results must not require human interpretation
  • Should be independent of other tests and their order of execution
  • Can serve as documentation of how (not) to use your code

Flaky Tests

  • Have different outcomes without changes to the code
  • Erode confidence in testing processes
  • Fix them (mute / skip them until they are fixed) or delete them

Software Quality

User's view (visible to the user)

  • Functionality
  • Reliability
  • Usability
  • Performance/ efficiency

Producer's view

  • Portability
  • Maintainability
  • Testability (vs. legacy code)
  • Transparency (is the code readable/ understandable?)

We Need a Unit Testing Framework

For writing, compiling and running unit tests, a unit testing framework is required.

There are many concurring unit testing frameworks for Python.

At the time of this writing, pytest offers the best features.

Example: Refactor a Function

def add_to(n):
    Return the sum of all values from 1 to n.
    sum = 0
    for i in range(n+1):
        sum += i
    return sum

After experiencing performance problems, we want to use a faster algorithm

# after refacturing
def add_to(n):
    Return the sum of all values from 1 to n.
    return (n / 2.0) * (n + 1)

How can we apply the change without risking to destroy our application?

  1. Define good test cases
  2. Implement test cases using your project's unit testing library
  3. Run the tests before the change to ensure they pass
  4. Apply the change, then run the tests again

1. Define Good Test Cases

Using pencil and paper / brainstorming: define relevant test cases

  • All code paths should be used
  • Add one or more common cases
  • Include relevant edge cases

25 is a possible common case. Add 26 as an even input. The edge cases should include 0 and 4294967295.
Expected results for these are 325, 351, 0 and 9223372034707292160.

Let's also add 500000000 (expected result 125000000250000000) as 4294967295 does not terminate with the current implementation.

2. Implement defined test cases

from before_refactoring import add_to
# from after_refactoring import add_to

def test_add_to():
    assert add_to(0) == 0
    assert add_to(25) == 325
    assert add_to(26) == 351
    assert add_to(500000000) == 125000000250000000

def test_add_to_slow():
    assert add_to(4294967295) == 9223372034707292160

3. Run the Unittests Before the Change

# takes rather long; can be interrupted via [Ctrl]+[C]
# tests are discovered automatically
pytest  # or `python -m pytest`

What to do now?

pytest --help
pytest --collect-only  # list all available tests
pytest -k "not slow"  # ignore tests with `slow` in their name

4. Apply the Change, then Run the Tests Again

For the sake of the presentation, instead of applying the change, we use another source file.

Adjust the test to import from the new implementation and run the tests.




pip install --user --upgrade pip
pip install --user --upgrade pytest

or use your operating system's package manager

sudo apt install python3-pytest


Comprehensive documentation can be found on

Test-driven development (TDD)

TDD is a methodology that emphasizes writing tests before the actual code

Helps ensure that code is tested thoroughly and avoids regressions

Repeat the cycle for each new piece of functionality

  1. Write a failing test that describes the desired functionality
  2. Write the minimum amount of code necessary to make the test pass
  3. Refactor the code to improve its structure and maintainability
  4. Make sure that all tests pass

Whenever you are tempted to type something into a print statement or a debugger expression, write it as a test instead.
– Martin Fowler

and feedback...