■ A.2—Parameterized Test Case overriding runTest()
4.9 Define a test suite in XML
◆ Problem
You want to externalize a data-driven test suite’s data in an XML document. To do so, you need to know how to create a TestSuite object that contains test objects corresponding to the “test” elements in your XML document.
◆ Background
JUnit’s design assumes that different tests test essentially different things. When we wrote about extracting a test fixture (recipe 3.4, “Factor out a test fixture”), we mentioned that of the three main parts of a test, it is usually only the first part that we duplicate among many tests: creating the objects to test. The other two parts—
invoking methods and checking the results—vary from test to test; if they didn’t, we would have to question why we have five tests when one might do an equally good job. This is the reason that, by default, each test is defined as a separate test method inside a test case class. This is certainly not the only way to write tests.
If you’re reading this recipe, then likely it is because you have a small suite of data-driven tests that you’d like to run. These tests all invoke the same methods on the same objects, but the input and output vary from test to test. You started writing a few test methods, factored out the common part of the test—the engine—
and you now have a bunch of test methods, each invoking the one method that executes the engine. Although you need only one method to implement the logic behind your test, you have written a number of test methods that simply invoke the engine with different parameters for the input and expected result. All that duplication! You feel that there must be a better way.
Don’t worry, because there is, and if you have chosen XML as the way to define your test case input and output, this recipe will show you how to integrate your XML document into a JUnit test.
134 CHAPTER 4 Managing test suites
◆ Recipe
The following is a sketch of the solution:
1 Create an XML document to store the data for your tests. The document structure includes one XML element for each test. Each test element includes an XML element for the input and another XML element for the expected result.
2 Create a Parameterized Test Case using the instructions in recipe 4.8,
“Build a data-driven test suite.” In the process, you will create a custom suite method.
3 Change your custom suite method so that it loads your XML document, reads the input and expected result for each test, and then calls your new constructor to create the corresponding test object. Your suite() method returns a TestSuite containing a test object for each test in your XML document.
In essence, you are going one step beyond the Parameterized Test Case by push- ing the test data out into a file—in this case, an XML document. Listing 4.6 shows an example of an XML document describing a test for the “allocate money” prob- lem we describe in recipe 4.8.
<tests name="Money Allocation Tests">
<test>
<input>
<amount-to-split>$1000.00</amount-to-split>
<number-of-ways>6</number-of-ways>
</input>
<expected-result>
<cut>
<amount>$166.67</amount>
<number>4</number>
</cut>
<cut>
<amount>$166.66</amount>
<number>2</number>
</cut>
</expected-result>
</test>
</tests>
Listing 4.6 Test data for splitting money n ways
The input for this test One test tag per test
The output for this test
135 Define a test suite in XML
This format should be simple enough to read, although the expected result may need some explanation. We expect $1,000 split 6 ways to result in 4 accounts with
$166.67 each and 2 accounts with $166.66 each. Because we need to give these concepts names for our XML document, we say that we expect 4 cuts of $166.67 each and 2 cuts of $166.66 each.
Our task is now to follow the earlier steps and create a Parameterized Test Case whose data comes from our XML document. Because the entire solution requires about 200 lines of code—well-factored, of course—we refer you to solu- tion A.1, “Define a test suite in XML,” to see how we parsed the XML document into test objects. The result is a suite of data-driven tests whose data source is an XML document.
◆ Discussion
Although we used XML to define our test data, you don’t need to use XML to employ this technique. Any text file or other external data format is equally good:
flat file, comma-separated values, or database tables. Ron Jeffries often uses a very simple file format for his Customer Tests: a kind of rudimentary structured text, with a section for the input, a section for the actions to execute, and then a section for the expected result. To see an example, consult Ron’s Adventures in C# series at www.xprogramming.com.27 The article “The First Customer Test” provides an example of writing a Customer Test using NUnit, a C# cousin of JUnit. Although the sample code is C#, most Java programmers do not have problems reading it.
The most important lesson about writing Customer Tests is this: you don’t need a framework; you need tests. The framework will evolve if you just start writing tests.
◆ Postscript
Not long after we wrote this recipe, we had the opportunity to work with Ward Cunningham’s Fit framework for the first time (http://fit.c2.com). It is a spread- sheet- or table-based way to write Customer Tests and may be the next big wave in testing. This is the framework we’ve been waiting for. Go check it out.
◆ Related
■ 3.4—Factor out a test fixture
■ 4.8—Build a data-driven test suite
■ A.1—Define a test suite in XML
27Either that or his book Extreme Programming Adventures in C# (Microsoft Press, 2004).
136
Working with test data
This chapter covers
■ Retrieving test data from system properties
■ Retrieving test data from environment variables
■ Retrieving test data from files on disk
■ Retrieving test data from a database
■ Managing test data using such tools as Ant, JUnitPP, and DbUnit
137 Working with test data
Object-oriented software deals with data and behavior. If you are going to test most software prior to releasing it to users or into a production environment, you need test data to simulate inputs that trigger behavior you expect to occur when the software works correctly. Software is commonly written for transforming input data into different output data. You cannot test whether the system outputs the correct output data, usually, without providing it with test input data. Software can also be designed to generate answers from digested input data or to manage data storage and retrieval. Obviously, testing these software behaviors require a lot of test data.
JUnit and its extensions are used to test software at a variety of scopes, not just the unit level. JUnit is great for traditional unit testing and Test-Driven Develop- ment at the unit scope. It is also good for Integration Testing, often up to the sub- system level. At higher integration levels and in testing complex subsystems, a JUnit extension such as Cactus or JUnitEE must be used. Often your organization has its own in-house test harness based on extensions of JUnit that can be used to test these higher integration levels. You need different amounts and types of test data, depending on whether you want to write Object Tests, Integration Tests, or End- to-End Tests. Keep in mind that the more of the system you test at once, the more you need to engage the real production system for both retrieving and restoring test data between runs. This adds complexity to both the tests and the test environ- ment and makes the task of writing and executing tests more difficult.
Test data should be specialized and designed to force the code under test into branches and states that are only tested when fed specific inputs. Working with test data includes parameterizing data in test cases, which means identifying data that changes from test to test and providing it externally. A simple approach to parameterizing involves abstracting test data into parameters that can be passed in, or retrieved dynamically, at runtime rather than hard coded directly into the test case classes. Parameterization is a good thing because it decouples your tests and your test data into separate concerns with separate alternatives and solutions available. Parameterized tests should rely on good, portable Java conventions and good practices and idioms for dynamically passing data to test cases at runtime. By the same token, another good practice is to avoid creating test harness infrastruc- ture until it is needed; at that point, making test data globally available—possibly even in the test case class itself—makes more sense. A range of gradations exists in between the two extremes of parameterization and global (class-level) definition.
Generating data dynamically (possibly large amounts of data, as in volume test- ing) and restoring the state of shared data between test runs are important tasks for some types of testing, especially database-related tests.
138 CHAPTER 5
Working with test data
If you have test data that does not need to be variable, you probably can hard code it in your tests. Otherwise, there are several good practices for parameterizing test data out of test cases. Some techniques use basic built-in Java features, such as sys- tem properties, command-line arguments to the JVM, the Java Properties API, or ResourceBundles. While parameterizing data and configuring test cases and suites with data are the main activities in working with test data, another important activ- ity is setting up and restoring data to its initial state between test cases or test runs, preferably with some degree of efficiency so that test run durations are kept to a minimum. This chapter offers techniques for resetting database data and other general kinds of data using DbUnit and the JUnit TestSetup Decorator.
Data can come in a variety of forms, from primitive Java data type variables and Strings, to XML files, to records in database tables. The nature of the data needed for testing is determined by the responsibilities of the classes under test. Once, when working on a commercial J2EE application server, we needed a great deal of test data in the form of EJBJARs, WARs, and EARs because we had to test applica- tion server responsibilities, such as the ability to deploy, undeploy, and redeploy J2EEEARs, WARs, and JARs. On another project, which involved human resources middleware applications, we needed test data in the form of large hierarchical data sets of managers and employees in a relational database. At both companies we needed different, smaller amounts of file-based test data to test service and application configurations, security settings, and utility classes. We also used envi- ronment variables and system properties to glue it all together and make test har- nesses and test suites portable and easy to run on different user environments, platforms, and operating systems. Setting up and maintaining different types of test data is an important and time-consuming investment for unit and integration testing large and complex systems. Using portable solutions for these concerns even at the level of individual test cases enforces best practices, which is especially important as you move up the scale from utility classes to small command-line applications, to middleware applications, application servers, and so on. This chap- ter covers working with data in a variety of formats including XML, properties files, text files, relational databases, and system and environment variables. And it cov- ers some best practices and useful JUnit extensions, such as JUnitPP and DbUnit, for working with databases and homegrown test data repositories.