◆ Problem
You would like to stop maintaining the list of tests in a test suite. You would rather have JUnit build the test suite from the tests in your test case class.
104 CHAPTER 4 Managing test suites
◆ Background
You may have read one of the first articles ever written about JUnit, entitled
“Test Infected: Programmers Love Writing Tests.”1 This article was written in 1999, a time when JUnit was still in its infancy. Some of the techniques in this article are outdated. In particular, the article shows the reader how to build a test suite from the tests in the current test case class. The article shows the reader the following example:
public static Test suite() {
TestSuite suite= new TestSuite();
suite.addTest(new MoneyTest("testEquals"));
suite.addTest(new MoneyTest("testSimpleAdd"));
return suite;
}
This builds a test suite containing two tests implemented by the methods test- Equals() and testSimpleAdd(). We call this manually building a test suite, or doing it by hand, because the programmer must remember to update this list of tests every time he adds a new test method. Before JUnit 2.0 this was the only way to build test suites, and if you last used JUnit “a long time ago”—back before ver- sion 2.0—then this may be the only way you know how to build a test suite.
It also may be that you learned to write test suites from another programmer who did not understand or know about how to let JUnit build test suites automati- cally. If you have been using this manual test suite–building technique, you recog- nize how error prone it can be: every time you add a test, you need to remember to update this suite() method. Removing or renaming a test is easier; at least JUnit fails at runtime, indicating that it cannot find the test. But if you add a test to your test case and not to your suite() method, then you may not notice that JUnit is not executing your test. This can lead to a false sense of progress, as you think your test passes, when JUnit is not executing it at all!
This recipe will help you avoid this unfortunate situation.
◆ Recipe
Since the advent of JUnit 2.0, the framework provides a way to build a test suite out of the test methods in your test case class. As long as you follow a few simple rules, JUnit will find your tests and execute them:
1 http://junit.sourceforge.net/doc/testinfected/testing.htm.
105 Let JUnit build your test suite
■ Your test methods must be instance-level, take no parameters, and return nothing. That is, you must declare them as public void testMethodName(). This is a general requirement for test methods in JUnit, whether you build the test suite yourself or let the framework do it, but it bears repeating.
■ The name of your test method must start with “test” (without the quotes), all lowercase.
Otherwise, there are no restrictions on how you declare your test methods. You may throw whatever exceptions you like, although you should understand how JUnit han- dles that (see recipe 2.8, “Test throwing the right exception,” for details). The code in your test can do whatever you need it to do, although it should follow the typical rhythm of a test: create some objects, invoke some methods, check the results.
Returning to the example in the “Test Infected” article, the authors have already named their test methods according to the rules. Although the article describes implementing the suite() method as follows, JUnit does even that auto- matically for you. If you omit the suite() method altogether, JUnit will build the
“default” suite as though you wrote this code. (It does the same work, but a little differently, which causes a minor difficulty that we describe in recipe 4.3.)
public static Test suite() {
return new TestSuite(MoneyTest.class);
}
We hope it is clear by now that you should always let JUnit build your test suite automatically. As you will see in other recipes, there are times when you need to build a suite by hand, but that should be the exception, rather than the rule. Let the framework do the heavy lifting.
◆ Discussion
JUnit uses Java reflection2 to build the default test suite at runtime. In particular, JUnit searches your test case class for declared methods that follow the rules we have outlined and then adds those tests to a test suite. You can think of it as gener- ating the “manual test suite” code at runtime. JUnit implements this feature in the class junit.framework.TestSuite. We summarize what each method does in table 4.1.
2 These are the classes in the standard Java package java.lang.reflect.
106 CHAPTER 4 Managing test suites
The way that JUnit builds the default test suite promotes writing isolated tests, a generally accepted good Programmer Testing practice. You ought not to rely on your tests executing in a particular order, nor should the failure of one test affect the outcome of the remaining tests in the suite. Commonly, programmers want to run a set of tests in a particular order because they share a common test fixture:
they set up some data, run multiple tests using that data, and then throw the data away. To do this, not only do you need to set up and tear down the test fixture at the right time, but you also need to ensure that the tests run in a prescribed order. If not, a future test will look for fixture data that is not there. Even if the tests execute in the proper order, a failing test using a shared fixture typically leaves the fixture in an unexpected state, rendering the remaining tests essentially useless. You can learn about Kent Beck’s early experience with automated tests in his discussion about the Test-Driven Development pattern he calls Isolated Test.3
He tells the story of long-running, GUI-based automated tests that his project ran every night. In the morning, Kent would see paper on his chair: either a single page saying, “Everything works”; or a stack of pages detailing the failures, one per failing test. He writes, “I noticed after a while that a huge stack of paper didn’t
Table 4.1 How JUnit builds the default test suite
TestSuite method What it does
Constructor TestSuite(Class)
• Verifies there is a public constructor
• Verifies the class itself is public
• Invokes addTestMethod() for each declared method in the class
• Extracts test methods for all tests in this class’s superclasses up to the top of the class hierarchy
• Issues a warning if there are no tests
addTestMethod() • Verifies that the method is a public test method by invoking isPublicTestMethod()
• Issues a warning if the test method is correctly named but not public
• Creates the Test object for the method and adds it to the suite, if the method is a valid test method
isPublicTest- Method()
• Determines whether the method is a test method by invoking isTestMethod()
• Determines whether the method is public
• Returns true only if the method is both public and a test method isTestMethod() • Verifies that the method has no parameters, no return type, and a
name starting with test
3 Kent Beck, Test-Driven Development: By Example (Addison-Wesley, 2002), page 125.
107 Collect a specific set of tests
usually mean a huge list of problems. More often it meant that one test had bro- ken early, leaving the system in an unpredictable state for the next test.” His con- clusion: “Tests should be able to ignore one another completely. If I had one test broken, I wanted one problem. If I had two tests broken, I wanted two problems.”
The goal is to make it easier to decide from the list of failing tests where the pro- duction code is broken. This is one philosophy that underlies JUnit’s design, and one of the ways in which JUnit works—subtly, mind you—to help you improve your design: writing isolated tests leads to highly cohesive, loosely coupled classes, a hallmark of good object-oriented design.
The last key point to have in mind is that the default test suite for a test case class includes more than just the test methods you have declared on that class.
JUnit also adds any valid test methods it finds on any superclass of your test case class—all the way up to the root of the class hierarchy.4 Although many beginning JUnit practitioners are surprised by this design choice, it is deliberate and makes sense: this is the mechanism that enables you to enforce an interface’s contract.
In other words, it allows you to execute the same tests on all implementations of an interface, or on all subclasses of an abstract class, such as a framework exten- sion point. For more information on building Abstract Test Cases, see recipe 2.6,
“Test an interface.”
◆ Related
■ 2.6—Test an interface
■ 2.8—Test throwing the right exception