Wednesday, February 14, 2007

JUnit Best Practices, Part 4 -- Don't Be Too Assertive

In a previous article we examined the pitfalls of invoking the method under test many times from the same JUnit test. As a counterpart to that article, let's look at the case where a JUnit test contains multiple assertions.

Assertions are the part of the JUnit test that actually determines whether or not your method under test worked as expected. With a simple method under test, it's relatively easy to make that decision with just one assertion, but when the method under test performs more complex tasks, it's tempting to add multiple assertions to your JUnit tests to verify several different behaviors at once. Despite appearances, however, this approach can actually increase your overall testing time and effort.

Let's look at an example. Suppose we have this method under test:


separator() scans its input String and fills the three StringBuffers with the vowels, consonants, and whitespace found therein. A JUnit test for this method might look like this:


This seems like a natural test to write, but the problem here is that JUnit tests are designed to fail as soon as the first assertion fails. In the worst case, when all three assertions would fail, you'll wind up with this scenario:

First test run
The first assertion fails.
Debug/fix the code.

Second test run
The second assertion fails.
Debug/fix the code.

Third test run
The third assertion fails.
Debug/fix the code.

Fourth test run
All assertions pass.

You've had to make three separate iterations of finding and fixing problems before your test passes. Since your brain is required for each debug/fix effort, this process cannot be automated. Instead it becomes time-consuming, frustrating, and inefficient. Worse, the likelihood of introducing a new bug increases every time the code is modified.

A better approach would be to split this one test method into three tests, each with a single assertion:


Now you can run your test suite once and identify all the problems before you dive into the code to fix them. True, your test suite has increased in size, from one test to three, but you've always had three tests all along anyway, and this change merely reorganizes your test suite to reflect that. More importantly, striving for one assertion per unit test streamlines the test-debug-retest cycle and improves the overall testing effort.

1 Comments:

Anonymous waxrx said...

I agree with your conclusion in general. However, in many real-world cases, the test set-up can be fairly complex, so that running two tests at once seems reasonable (e.g., assert not null, assert equals).

Wed Sep 26, 09:28:00 AM 2007  

Post a Comment

<< Home