■ 4.5—Scan the file system for tests
4.2 Collect a specific set of tests
◆ Problem
You cannot use (or do not wish to use) the default test suite that JUnit creates from your test case class. You want to ignore certain tests or have control over the order in which JUnit executes the test, if not both.
4 Strictly speaking, that would be the class Object, but for our purposes, we mean the class TestCase.
108 CHAPTER 4 Managing test suites
◆ Background
There are a number of good reasons to want to manually choose which tests go into the test suite for your test case class. Unfortunately, there are also a number of not-so-good reasons to want to do this, which is why we highly recommend read- ing the Discussion section of recipe 4.1, “Let JUnit build your test suite,” before you read any further. If you are reading on, then either you remain unconvinced by our recommendations or you really do need to build your test suite by hand.
JUnit practitioners—in particular, the ones who also do Test-Driven Develop- ment—have the tendency to be dogmatic in their approach to writing tests. Look- ing to Kent Beck as a leader, we are told repeatedly how important it is to write isolated tests, as doing so forces us to work that little bit harder to improve our design. This works wonders when we are able to practice Test-Driven Develop- ment—that is, when the production code under test has not yet been written. In a legacy code environment, though, you do not have the luxury of a testable design.
It is hard enough to achieve 100% test isolation in a well-designed system; it is generally impossible to do so in a system designed entirely without testing in mind. You may well be one of the lucky people charged with testing such a system, and if you are, then you almost certainly need to read this recipe, in addition to most of the rest of the recipes in this chapter. They provide instructions on con- trolling the order of tests in a test suite so that you can build a safety net to help you move toward 100% test isolation.
◆ Recipe
To build your test suite by hand, do the following:
1 Create a method on your test case class called suite(). It must be declared as public static Test suite(), taking no parameters and returning a Test object.5
2 In the body of your custom suite method, create a TestSuite object, and then add Test objects to the suite: either individual tests or other test suites. Return this TestSuite object.
That is a sketch of how to build the test suite by hand. We offer an example from
“Test Infected”:
5 It actually returns a TestSuite object, but remember that TestSuite implements Test.
109 Collect a specific set of tests
public static Test suite() {
TestSuite suite= new TestSuite();
suite.addTest(new MoneyTest("testMoneyEquals"));
suite.addTest(new MoneyTest("testBagEquals"));
suite.addTest(new MoneyTest("testSimpleAdd"));
suite.addTest(new MoneyTest("testMixedSimpleAdd"));
suite.addTest(new MoneyTest("testBagSimpleAdd"));
suite.addTest(new MoneyTest("testSimpleBagAdd"));
suite.addTest(new MoneyTest("testBagBagAdd"));
return suite;
}
The order in which you specify these tests is the order in which JUnit executes them, if test order is important to you.
◆ Discussion
To understand this code in more depth, first note (or remember, if you already know this) that each test runs in its own instance of your test case class. This is one of the fundamental aspects of JUnit’s design: it is the way that JUnit encourages test isolation. This is also the reason that we implement test fixture objects as instance-level fields on the test case class: each test is an instance of the test case class, so it has its own copy of the fixture. The constructor MoneyTest(String) takes as its parameter the name of the method that JUnit should call to execute the test. This constructor creates an instance of the test fixture (an isolated copy of the fixture) that invokes the specified method to run the actual test. When the framework executes this test, it eventually invokes the method TestCase.run- Test(), which uses reflection (yet again!) to invoke the method whose name you provided to the test case class’s constructor.6
The method TestSuite.addTest() simply encapsulates an internal collection—
a List, in particular—of Test objects. When the framework runs the test suite, it runs each Test object in that suite in the order in which they appear in the list.
Most of those Test objects are single tests, but JUnit allows you to build bigger suites out of smaller suites, so you can add either a TestCase or a TestSuite when you call addTest(). For an example of building a suite of suites, see recipe 4.3,
“Collect all the tests in a package.”
There is one simple consequence of the way JUnit lets you build a test suite that not many practitioners take advantage of. We do not know whether this is a use-
6 This method also verifies that the test is a valid test method. Open the source for junit.frame- work.TestCase for more detail. There is no better documentation than the source code.
110 CHAPTER 4 Managing test suites
less feature or simply a large, collective blind spot.7 Look back at the suite() method and notice one kind of duplication: the class name MoneyTest. This test suite is a collection of tests coming from the same test case class. If we specify some other test case class than MoneyTest, we can collect tests from a diverse set of test case classes. You may now ask yourself, “Why tell us this? What good is it?”
Frankly, we have not leveraged this fact in any meaningful way in our past work with JUnit, but there is a particular scenario that we feel puts this feature to good use. It’s nothing obscure: you can use it in the course of investigating defect reports coming from outside the programming team.
One of the recommendations that the Test-Driven Development community makes concerns what to do when someone else finds a defect in your team’s code.8TDD practitioners would say that because someone else has found a defect, you have either missed a test that you should have written or written a test incor- rectly. Often you have decided on an expected result that is unexpected for your users, so although your code passes your tests, the end users see behavior they do not expect. In this case, TDD recommends that you do as follows:
1 Identify the Customer Test (or functional test, or acceptance test) that does not match the expectations of the end users.
2 Use the Customer Test to help you determine which objects are involved in making that Customer Test pass, and then identify the Programmer Test that you either got wrong or missed altogether.
3 Fix (or write) the Programmer Test, make it pass, and then fix the Cus- tomer Test in the corresponding way. It may already pass as a result of your code changes, but if it does not, tie up whatever loose ends you may have to make the Customer Test pass.
At this point you have zero failing tests and can rerelease your software complete with a fix for this defect.
Now, what does this have to do with building a suite of JUnit tests?9
As you identify the Programmer Tests that match the failing Customer Test, we recommend creating a temporary suite for those tests. This test suite is an object- level view of the behavior the Customer Test verifies. As you work on the problem, execute the Programmer Test suite repeatedly as you go. This keeps you focused
7 We suspect the feature is quite useful, but we may forget to use it the next time we have the opportunity.
8 We think it couldn’t happen to us.
9 We try to get to the point quickly; we really do.
TE AM FL Y
Team-Fly®
111 Collect all the tests in a package
on fixing the problem at hand, while giving you object-level feedback on the effects of your changes. This is an alternative to running all the system’s tests after every change, which will cause more changes—changes that you may not be ready to make yet—to ripple out and thus distract you from the work at hand. This Pro- grammer Test suite may include tests from a number of different test case classes throughout the system, and JUnit allows you to build a test suite from an arbitrary slice of the system’s entire collection of tests. That is quite handy.
After reading this, you might think it a good idea to start building these Pro- grammer Test suites as you write (or receive) each Customer Test. We recom- mend against it: the relationship between a Customer Test and its constituent Programmer Tests may change a great deal over time, and the cost of maintaining those relationships in code is more than the time you might save discovering those relationships when required. Either way, we have described a typical sce- nario in which you may want to build a test suite from an arbitrary collection of tests.
◆ Related
■ 4.1—Let JUnit build your test suite