◆ Problem
Your testing environment is expensive and complex to set up. You need to do it once before you can run tests, but you do not need to (or cannot afford to) set it up before each test run. You would like to perform this environment setup only once.
◆ Background
Rakesh Madhwani provided the motivation for the recipe in the JUnit Yahoo!
group. His problem was simple: he wanted to be able to execute a large test suite against a graphical user interface, but he did not want to start up and shut down the GUI for each test. He did not want to even start it up and shut it down for each test suite run. He wanted to launch the GUI once, execute the tests as many times as he liked, however he liked, and then shut the GUI down once when he had fin- ished. His first try was using the TestSetup Decorator, but the TestSetup Decorator
165 Perform environment setup once
for multiple test runs
only provides support for one-time setup before and one-time teardown after each test suite run. What he wanted was one step further back: one-time setup before launching the test runner and one-time teardown when he finished using the test runner. This recipe provides various solutions to this problem, depending on which of the test runners you use and what kind of setup you need.
◆ Recipe
The approach you take to implement this one-time setup depends on any con- straints you may have on how you execute tests, in general. Different tools admit different solutions to this problem. The solution also depends on the nature of your environment setup tasks. First, we present a summary of the various approaches in table 5.1, and then we describe each approach in turn.
Initializing data or services outside the Java Virtual Machine
You may need to start services outside your Java application, such as database serv- ers, web servers, and so on. These kinds of tasks fall outside the purview of your tests, so it is best to configure your operating system, when possible, to start these services automatically. If you are unable to do this, then we recommend writing either shell scripts or an Ant target that starts all the various services you need before executing your tests. If you can configure the operating system to execute this script on startup, that would be great; otherwise, you really do have to simply depend on yourself to remember to perform these steps before executing your tests. The bad news is that you are not the only person who will ever need to run your tests, so what about educating others?
If you treat your test execution environment like a large tool (such as your IDE or a web browser), then it is logical to tell people, “Here is how you start the test envi- ronment. Make sure you do this before trying to execute any tests.” It may be most effective to provide this information in the form of online or hard copy documenta- tion. We hope your environment is not so complex that you need a user’s guide just to execute tests, but we recognize that it happens, having seen it ourselves. It is
Table 5.1 Summary of approaches to performing one-time environment setup
Setup tasks / Tool Text-based test runner Swing-based test runner Initialize data or services
outside the Java Virtual Machine.
Perform the one-time setup and tear- down tasks manually as needed, or use a shell script.
Override the terminate() method to invoke your teardown code. Invoke your setup code before launching the test runner.
Initialize objects inside the Java Virtual Machine.
Nothing you can do: each test suite run occurs in a separate JVM.
166 CHAPTER 5
Working with test data
generally impossible to remove the complexity altogether, but it is usually possible to reduce the complexity to “pushing a single button” with some clever automation.
NOTE A one-button test environment—If you think your test environment is too complex to automate, consider this story from our experience. J. B. worked in an organization that used VisualAge for Java as its development envi- ronment. The product we were building was made up of over 40 different projects on the workspace at once. Despite varying degrees of effort to reduce dependency among these projects, it was generally impossible to remove anything from the workspace and expect the overall product to work. In addition to this code complexity, the product required a data- base of some 200 tables and thousands of rows of startup data and test data. This was a complex environment.
After some time, a group decided to build a standard set of instruc- tions to help a programmer install and set up this environment on her workstation. The instructions ran several pages and included close to 50 manual instructions. Although workable, it generally took over 30 min- utes of attentive effort to set up the test environment, and it did not always work. Sometimes a programmer would go days without being able to write code. Finally, one programmer took matters into his own hands and used Microsoft’s ScriptIt (which Microsoft has since retired) to auto- mate VisualAge for Java. He crafted a series of scripts and worked with the organization’s build team to automate the entire process of down- loading code, importing it into VisualAge for Java, creating the database, and importing all the necessary data. The result was literally a single push of a button, resulting in a working test environment in approximately 25 minutes, every time. Although it was not much faster than the manual process, it could run unattended...and it worked!
Remember this story the next time someone complains that there is
“no way” to automate your test environment.
Initializing Java objects within the Java Virtual Machine
You may need to set up some global data for your tests: data that is rather expen- sive to set up. So expensive, in fact, that you cannot afford to set it up per test—or even per test run! We highly recommend looking into changing this situation, because it appears to indicate some fairly serious design problems, but if you have to get something running now, then what you do depends on how you run your tests.
If you use the text-based test runner on its own, then we are sorry to say that you are out of luck. The text-based test runner executes each test suite in its own JVM, so you have to perform the one-time setup and teardown each time you exe- cute a test suite. The good news is that you can automate this by creating your own
167 Perform environment setup once
for multiple test runs
simple custom test runner. The following code simply invokes its own setUp() before invoking the text-based test runner, and then invokes tearDown() afterward:
package junit.cookbook.running.test;
import junit.framework.*;
import junit.textui.TestRunner;
public class TextBasedOneTimeEnvironmentSetupTestRunner { public static Test suite() {
TestSuite suite = new TestSuite();
// Create your test suite...
return suite;
}
private static void oneTimeEnvironmentSetUp() { System.out.println("Setup");
}
private static void oneTimeEnvironmentTearDown() { System.out.println("Teardown");
}
public static void main(String[] args) throws Exception { oneTimeEnvironmentSetUp();
TestRunner.run(suite());
oneTimeEnvironmentTearDown();
} }
In order to execute test suites many times in the same JVM, you need to use one of the graphical test runners, such as the Swing-based test runner. You could customize its behavior in a manner similar to how we added one-time setup and teardown to the text-based test runner, but as you will see, it is not quite so simple. If you try to copy the code from the previous example (changing junit.textui.TestRunner to junit.swingui.TestRunner), then your setup code executes as expected, but your teardown code executes before you close the test runner. In particular, this is what happens:
1 The JVM executes your setup code.
2 The JVM launches the test runner and executes the test suite.
3 The JVM executes your teardown code, leaving the test runner open and waiting for you to press Run again.
At this point, if you execute another test suite (possibly the same one), it may execute while your custom teardown code is also executing, which can only create a big mess.
168 CHAPTER 5
Working with test data
You need a way for the test runner to wait until you press Exit before executing your teardown code. Fortunately, that is not difficult at all.
The Swing-based test runner declares the method terminate(), which it invokes when you press Exit or close the Test Runner window. This is the hook you need to execute your teardown code. Simply override terminate() and have it invoke your teardown code before invoking its superclass’s implementation. Here is the complete code:
package junit.cookbook.running.test;
import junit.swingui.TestRunner;
public class SwingBasedOneTimeEnvironmentSetupTestRunner { private static void oneTimeEnvironmentTearDown() { System.out.println("Teardown");
}
private static void oneTimeEnvironmentSetUp() { System.out.println("Setup");
}
public static void main(String[] args) throws Exception { oneTimeEnvironmentSetUp();
TestRunner testRunner = new TestRunner() { public void terminate() {
oneTimeEnvironmentTearDown();
super.terminate();
} };
testRunner.start(new String[] { "com.mycom.MyTestSuiteClass" });
} }
We bring one difference to your attention between this class and the one that uses the text-based test runner: the Swing-based test runner does not provide the same run() method that the text-based test runner provides, so it is necessary to pass in the fully qualified name of the test case class to execute. This is a minor annoy- ance, but not a serious roadblock.
◆ Discussion
One-time environment setup is a serious smell in any application, particularly if it is necessary to initialize data within the JVM before executing your first test. This is a sign that objects are relying on global data, which is not only bad for testing, but
169 Perform environment setup once
for multiple test runs
makes for a rigid, inflexible design. This is the kind of practice that drives up the cost of change. We strongly recommend that you take steps to reduce an object’s dependence on global data, preferring instead to have that data provided to the object through its constructor. A particular case of this design problem deals with Singletons, which cause all manner of testing difficulties and which we discuss in more detail in chapter 14, “Testing Design Patterns.”
Not long after we wrote this recipe, Rakesh grafted a solution onto the text- based test runner that addresses the more specific problem of running the same test multiple times with one-time setup. We include that solution here as a viable alternative for that problem. Create your own subclass of junit.textui.TestRun- ner and override this method:
protected TestResult start(String args[]) throws Exception { while(true) {
result = doRun(getTest(testCase), wait);
}
return result;
}
This executes the test suite repeatedly until you stop the test runner by pressing the “break” button (CTRL+C, for example). Although this is one way to solve the problem, it has some drawbacks that we recommend you take into account before using it yourself:
■ The tests run in an infinite loop, which means that you can never use this test runner as part of an automatic build-and-test process. Because that is not its purpose, there is little cause for concern.
■ This solution duplicates some of the logic in the superclass, meaning that changes to the superclass (JUnit’s text-based test runner) could affect the correctness of this solution. You need to remember this if you upgrade JUnit.
If none of these problems worry you, then this solution might just be simple enough to work!
◆ Related
■ Chapter 6—Running JUnit Tests
■ Chapter 14—Testing Design Patterns
170 CHAPTER 5
Working with test data