◆ Problem
You have written a test by overriding the method runTest(), rather than letting JUnit identify your test methods. Now JUnit sets up your test fixture only once before all the tests, rather than once before each test.
◆ Background
There are a number of reasons to override runTest(), not the least of which is implementing a Parameterized Test Case. We described this technique in recipe 4.8,
“Build a data-driven test suite.” There we suggested overriding runTest() so that you could give each of your parameterized tests a meaningful name rather than accepting the default convention of naming the test after the method that imple- ments it.6 Here you have delved into the inner workings of the framework, with intent to solve a particular problem. We understand that not all forays into the bowels of a framework are done with intent. Sometimes they are done out of des- peration, or even just because you didn’t quite know what you were doing.
Don’t feel bad: we celebrate our ignorance. That might sound strange; good programmers know their limitations and don’t feel bad asking for help when they find themselves in over their heads. This is what keeps a programmer from being
5 In particular, Laurie Williams and Robert Kessler, Pair Programming Illuminated (Boston: Addison-Wesley Professional, 2003)
6 In a Parameterized Test Case, each test invokes the same method, but with different parameters. If you used the default naming convention, then it would be difficult to differentiate these tests from one another.
242 CHAPTER 8
Troubleshooting JUnit
holed up in a room for days, “going dark;” that is, silently making no progress because he doesn’t understand some aspect of what he’s doing. Faced with an unfamiliar task, we think it’s important to put all our energy into finding a good solution, even if it is not our own solution. Don’t hide what you don’t know: get it out in the open, find someone who does know, then learn from that person. In the long run, you’ll feel better.
Now what does this have to do with your problem?
It is common for a novice JUnit practitioner to expect the setUp() method to be invoked once for the entire test suite, rather than once for each test. She has the same expectation for tearDown(). This misunderstanding generally comes from confusing the TestCase class with an instance of the test case class. Although you include many test methods on the TestCase class, JUnit executes each method in a separate instance of the test case class, invoking setUp(), the test method, then tearDown(), in that order. If no one has explained this to you, though, then in your quest to get a test to run—any test—you may poke around in the dark a little, stumble upon overriding runTest(), get that to work, and claim victory. After all, the “Test Infected” article shows overriding runTest() to write a test, so that must be the proper technique!
Once you have one test working, you don’t see how to add the second test, so you write two methods, invoke them both from runTest() and it continues to work. Your tests execute and that’s all that matters. Unfortunately, as soon as you try to factor out a common fixture, you will find yourself in trouble, and that is what might have led you here. We think this recipe can help you.
◆ Recipe
When you decide to override runTest() it is important that you understand that JUnit treats whatever is in runTest() as a single test. This means that if you invoke multiple methods from inside runTest(), expecting each to be its own test, you’re in for a surprise, because JUnit treats them all as a single test. Because JUnit invokes setUp() and tearDown() once per test, it only invokes them once each, and not once for each method you invoke from within runTest().
Most of the time—including the case in which you are just trying to execute many tests in a single fixture (test case class)—you do not need to override run- Test(), so we recommend you do three things:
1 Create a separate test method for each test you wish to execute.
2 Use the standard JUnit naming rules for your test methods.
3 Do not override runTest().
243 Test setup fails
after overriding runTest()
JUnit will create the default test suite from your test methods, executing setUp() before each test and tearDown() after each test, just as you were hoping it would do. If you want to execute multiple tests in a single fixture, then you certainly do not want to override runTest(). Instead, implement each test as a method follow- ing the usual JUnit guidelines, which we describe in recipe 8.1, “JUnit cannot find your tests.”
◆ Discussion
Here is the method junit.framework.TestResult.runBare(), which executes your test:
public void runBare() throws Throwable { setUp();
try {
runTest();
}
finally { tearDown();
} }
As you can see, when JUnit executes your test, it first invokes setUp(), then your test method, then tearDown(). As we wrote previously, JUnit treats whatever code you write inside runTest() as a single test. By default, runTest() uses reflection to invoke one of your test methods.7 JUnit expects runTest() to execute a single test.
Now that you’ve seen how JUnit invokes runTest(), you can see the effects of overriding it in your test case class. You will not need to do it often, but when you do, you’ll know how it fits into the rest of the JUnit framework.
NOTE There are times when it is necessary to override runTest(), such as when implementing a Parameterized Test Case (see recipe 4.8, “Build a data- driven test suite”). Typically such a test strategy involves executing a sin- gle test for multiple fixtures, rather than multiple tests for a single fix- ture, which is JUnit’s normal mode of operation. If you are building a Parameterized Test Case, then it is unlikely that you will run into the problem this recipe is meant to solve.
◆ Related
■ 4.8—Build a data-driven test suite
■ 8.3—JUnit does not set up your test fixture
7 In other words, that is how TestCase implements runTest().
244 CHAPTER 8
Troubleshooting JUnit