◆ Problem
You want to build a suite that runs the same test many times with different data.
◆ Background
The obvious way to do this with JUnit is simple, but tedious. Let us describe how we first experienced this tedium.
We had a method we wanted to test, so we wrote a test. This first test verified a simple case and was easy both to write and to make pass. The next test tried a slightly more complex case, but it too was simple to write and make pass. At that point we had two test methods that looked very similar. The next test tried a boundary condi- tion, where the output was a little different. We wrote this test, made it pass, and then saw the amount of duplication in our tests. All three tests did essentially the same thing: they invoked the same methods on the same kinds of objects. The only differences were the method parameters and the expected results. The names of the methods even showed some duplication: each test method’s name had both the name of the method we were testing and the particular condition we were testing.
We wanted to do something about all this duplication.
Being good little programmers, we factored out the common parts of the tests into a method that performed the core of the test by accepting the input and expected result as parameters. We called this method doTest() to indicate that it was actually performing the test. We changed all the JUnit test methods to call doTest() with the various parameters. This solution pleased us, but every time we wanted to test a new boundary condition, we had to add a test method that did nothing more than invoke doTest() with a new set of parameters. If all we’re doing is adding new test data, why are we writing code? We should just be adding data somewhere.
This recipe describes how to parameterize a test case so that adding new tests is as simple as adding the new test data. These kinds of tests are generally referred to as data-driven tests.
128 CHAPTER 4 Managing test suites
◆ Recipe
First, you need to build a Parameterized Test Case. This is a test case class that defines a single test method but includes a custom suite method that provides the data for each test. The suite method builds a suite of tests, each test executing the same test method but providing a different test fixture for each instance of the test. The fixture contains the input to the test and the expected results. The end result is a suite of test objects, each of whose test method is the same, but whose test fixture is different. JUnit can then execute the resulting test suite, executing each test on its own, different fixture. Here is a sketch of how to transform a gar- den variety test case class into a Parameterized Test Case:
1 If you have not done so already, factor out the common parts of your test—what we call the engine of the test—into its own method. Call the method doTest(). Its parameters will be the input to the method under test and the result you expect.
2 Change each test method to call doTest() with the appropriate parameters.
3 At this point, you have a test case class whose test methods all delegate to doTest() and otherwise only specify the input and expected result for each test. Factor that data out into a test fixture.
4 Create a constructor for your test case class whose parameter list matches the parameter list of doTest(). Your constructor should call super ("doTest") and then store the parameters in instance-level fields. For sim- plicity, name the fields the same as the parameters to doTest().
5 Create a suite() method to build a custom test suite.
6 For each test method in your test case class, take the parameters you pass to doTest() and use them to create an instance of your test case class. Add this test object to the test suite. Remove the test method.
7 Once all the test methods have been removed, remove the parameter list from the method doTest(). It should now resemble a test method in that it is public, it occurs at the instance-level, it has no parameters, and it has no return value. Because you have added fields with the same name as the parameters to doTest(), the method should still compile.
8 Consider renaming doTest() to something that better reveals the intent of the test. Remember to change the first line of the new constructor to match the new name of this method.
129 Build a data-driven test suite
You end up with a test suite created from test objects that correspond to the test methods you had written by hand before. Your test data was formerly scattered among the various test methods, but now it is found in one place: your new suite() method. This makes it easier to see at a glance all the data you use in your test suite.
To illustrate this technique, let us return to our Money class and consider a problem that Martin Fowler raises in Patterns of Enterprise Application Architecture:
allocating an amount of money evenly to multiple accounts. Although splitting money n ways seems like a simple problem, people tend to be very concerned about rounding problems when their money is involved, so it is important to get this algorithm right.24 The essence of the problem is that blindly rounding off to the nearest penny either loses money or creates it out of thin air, neither of which makes everyone happy. Because concrete examples are always better, consider how $1,000 should be split 6 ways: 2 of every 3 accounts receives $166.67, the other $166.66. In total, 4 accounts should receive $166.67 each and 2 accounts
$166.66 each. The total remains exactly $1,000. Blindly rounding would have cre- ated two cents where there were none.25
You will want to have a number of these tests, with different amounts to split and different ways to split them. Each test does essentially the same thing: take an amount of money, split it some number of ways, and then check that the accounts received a fair “cut” of the money. The engines of the tests are the same, but the input and expected output are different; so we push the data into our custom suite() method, as you can see in listing 4.5.
public class AllocateMoneyTest extends TestCase { private Money amountToSplit;
private int nWays;
private Map expectedCuts;
public AllocateMoneyTest(
Money amountToSplit, int nWays,
Map expectedCuts) {
super("testAllocate");
24There are a surprising number of ways to get it wrong.
25Printing money causes inflation, which leads to the overall collapse of the economy. Programmers can do real damage when they put their minds to it.
Listing 4.5 AllocateMoneyTest
Must match test method name The test parameters become the fixture
130 CHAPTER 4 Managing test suites
this.amountToSplit = amountToSplit;
this.nWays = nWays;
this.expectedCuts = expectedCuts;
}
public static Test suite() throws Exception { TestSuite suite = new TestSuite();
Map oneGSixWays = new HashMap();
oneGSixWays.put(new Money(166, 66), new Integer(2));
oneGSixWays.put(new Money(166, 67), new Integer(4));
suite.addTest(
new AllocateMoneyTest(
new Money(1000, 0), 6,
oneGSixWays));
Map oneGTwoWays =
Collections.singletonMap(
new Money(500, 0), new Integer(2));
suite.addTest(
new AllocateMoneyTest(
new Money(1000, 0), 2,
oneGTwoWays));
return suite;
}
public void testAllocate() {
List allocatedAmounts = amountToSplit.split(nWays);
Map actualCuts = organizeIntoBag(allocatedAmounts);
assertEquals(expectedCuts, actualCuts);
}
// A bag is a collection of objects that counts the // number of copies it has of each object.
// The map's keys are the objects and the values are // the number of copies of that object.
private Map organizeIntoBag(List allocatedAmounts) { Map bagOfCuts = new HashMap();
for (Iterator i = allocatedAmounts.iterator();
i.hasNext();
) {
Money eachAmount = (Money) i.next();
incrementCountForCutAmount(bagOfCuts, eachAmount);
}
return bagOfCuts;
}
Creates the first test
Creates the second test
TE AM FL Y
Team-Fly®
131 Build a data-driven test suite
private void incrementCountForCutAmount(
Map bagOfCuts, Money eachAmount) {
Object cutsForAmountAsObject = bagOfCuts.get(eachAmount);
int cutsForAmount;
if (cutsForAmountAsObject == null) { cutsForAmount = 0;
} else {
cutsForAmount =
((Integer) cutsForAmountAsObject).intValue();
}
bagOfCuts.put(
eachAmount,
new Integer(cutsForAmount + 1));
} }
The key points here are that you have created a test fixture from the old test parameters, your constructor must specify the test method name when calling super(), and you create a test object for each group of data you’d like to use in your test. These are the features of a Parameterized Test Case.
◆ Discussion
Note that although we named our test method testAllocate(), following the JUnit naming rules, we did so as a matter of convention rather than out of necessity.
Because we built the test suite ourselves, we could have named the method any way we like, as long as it was otherwise a valid test method (meaning it is public, occurs at the instance-level, has no parameters, and has no return value). What is important is that the name you pass into the test case class’s constructor is the name of your test method. JUnit still uses reflection to execute your test method. Unless you have good reason to change the name, we recommend following the conven- tion. It is simply easier for everyone that way.
One drawback to this approach is that although each test operates on different data, all the tests in your suite will have the same name. This can make it difficult to determine exactly which test is failing from JUnit’s output alone. One simple solution is to add the test input into the failure messages for each assertion in the test. This would work, but it is better to have meaningful names. You can achieve
132 CHAPTER 4 Managing test suites
this by overriding runTest() and placing your test code directly inside that method. This way you sidestep JUnit’s default behavior, which is to invoke a method whose name matches the name of the test case.26 If you override runTest(), you can pass a more meaningful name as a parameter to the constructor of your test case class. For a complete example, refer to solution A.2, “Parameterized Test Case overriding runTest().”
The method you want to test may have a few different ways to respond to its input, depending on whether the input is valid. It is common, for example, for a method to process valid input but throw an exception on invalid input. In this case, you need to change your Parameterized Test Case a little. Build one test method for each major kind of behavior: one test verifies that the object under test correctly processes valid input; the other test verifies that the object throws the correct exception when it receives invalid input (see recipe 2.8, “Test throw- ing the right exception”). When you build the test suite, remember to choose the appropriate test method based on the input you plan to pass in—valid or invalid.
You may decide to implement these different behaviors in separate Parameterized Test Cases, but if there are only a few data sets, it may be simpler to combine the two into a single test case class. As always, experiment with each solution and com- pare the results.
It is common to move the test data to a file, a database, or some other location away from the test code. This makes adding new test cases simpler because it avoids rebuilding Java code. For an example of externalizing test data to an XML document, see recipe 4.9, “Define a test suite in XML.”
NOTE Don’t try this with Cactus!—If you are using Cactus for in-container J2EE testing (see chapter 11, “Testing Enterprise JavaBeans”), then you cannot employ this technique to build a data-driven test suite. Cactus instantiates TestCase objects when executing tests on the server, which means that although you may be passing in test fixture objects on the client side, Cactus does not pass those fixture objects into the server-side test, render- ing that fixture data useless for in-container testing. Your only recourse is to extract a method containing the test logic, such as the doTest() method from this recipe, and then write a test method for each combina- tion of parameters that invokes doTest(). The main drawback is that adding new test data requires changing Java code rather than just data.
You can probably generate the Parameterized Test Case class source code using something like the Velocity template engine, but we recommend
26This is part of the machinery that performs automatic test suite extraction from your test case class.
133 Define a test suite in XML
doing so only after you have analyzed the trade-off between manually maintaining the test and building the code generator (and then integrat- ing it into your build and test environment).
◆ Related
■ 2.8—Test throwing the right exception