◆ Problem
You want to write isolated tests for code that writes to the file system, so you need to clean up the file system between tests, but that does not appear to work as you would expect.
◆ Background
When a Java application writes to the file system using the file I/O libraries, file system changes occur asynchronously relative to the Java application. In other words, the Java application does not wait for the file system operations to com- plete. Ordinarily, the file system operation takes so little time to complete that this slight drift in execution time has no impact on your application; however, it easily affects well-isolated tests.
Consider a test fixture whose tearDown() method cleans up the directory you used for the test. (Which directory that is does not concern us for the moment.) The fixture is something similar to that found in listing 17.1.
public class FileSystemOutputStrategyTest extends TestCase { private File expectedOutputFile;
protected void setUp() throws Exception {
expectedOutputFile = new File("./test/output/a/b/c/d.html");
}
public void testWriteOutputToDirectory() throws Exception { // Try writing output to file
// Verify the file exists }
protected void tearDown() throws Exception { if (expectedOutputFile.exists())
expectedOutputFile.delete();
} }
When you have about five tests using this fixture, you begin to notice that the file created in test #3 has not been deleted by the time test #4 executes, causing the latter to fail. You do not have the test isolation you think you have.
The next impulse is to clean up the file in both setUp() and tearDown(), which is not a bad idea, anyway. Listing 17.2 shows the new code.
Listing 17.1 A fixture that tries to delete its files
delete() complains when a file does not exist
606 CHAPTER 17 Odds and ends
public class FileSystemOutputStrategyTest extends TestCase { private File expectedOutputFile;
protected void setUp() throws Exception {
expectedOutputFile = new File("./test/output/a/b/c/d.html");
if (expectedOutputFile.exists()) expectedOutputFile.delete();
}
public void testWriteOutputToDirectory() throws Exception { // Try writing output to file
// Verify the file exists }
protected void tearDown() throws Exception { if (expectedOutputFile.exists())
expectedOutputFile.delete();
} }
This does not seem to solve the problem, particularly if your tests are short and execute quickly. Anything else seems like a hack, so what do you do?
◆ Recipe
Sadly, we only see two options.
1 Change the production code to eliminate the file system from the equation, a strategy we discuss in recipe 17.2, “Test your file-based application without the file system.”
2 Slow the tests down.
That’s right, folks: slow the tests down. It pains us even to type those words.
If your test involves asynchronous communication—such as invoking file sys- tem operations—and it does not stop to acknowledge the other party completing their part of the test, then you need to slow the test down. This is what we mean:
protected void tearDown() throws Exception { if (expectedOutputFile.exists()) { expectedOutputFile.delete();
pauseSoTheFileHasTimeToDelete();
} }
Listing 17.2 Cleaning up the file in setUp() and teardown()
Ugh! Duplication Ugh! Duplication
607 Clean up the file system between tests
You can implement this pause method with something as simple as Thread.sleep(250)—sleep for 250 milliseconds. Although this might work, there are a few drawbacks. First, the time to sleep depends on several factors, including CPU speed, hard drive speed, file size, and disk fragmentation. (This is not an exhaustive list, either.) Next, this pause slows your test suite down, discouraging you from executing it as often as you otherwise would. Finally, the decision to pause or not to pause has nothing to do with the test! It is a constraint of your produc- tion code’s implementation details. It ought to be irrelevant. Now if you are test- ing code that you built strictly to use with a file system, such as Prevayler (www.prevayler.org), then the rules are different. For most business applications, though, the file system is an incidental tool and not an integral part of the archi- tecture. Your tests ought not to have to worry about these low-level details. This is why we recommend factoring out the file system.
Nevertheless, if you inherit legacy code coupled to the file system and you need to add tests—possibly to enable future refactoring—then you need to apply this little hack until you can move to something better. This is another of those recipes that is good to know, not because you want to apply the technique, but because you might be forced to apply it at some point so that you can eventually phase out the need for it, similar to recipe 4.7, “Control the order of some of your tests.”
◆ Discussion
If you apply this technique, you need to be aware of the possibility for spurious
“false failures.” That is, from time to time your file system tests fail only because tearDown() did not pause long enough before the next test began to execute.
This wastes time in two ways: either you will carefully investigate each such failure and find it was a false alarm or, much worse, you will begin to ignore those failures and decide not to investigate a real defect in the same part of the code. One of the goals of Programmer Testing is to minimize the mean time between injecting a defect and discovering it. If you ignore failures, you defeat this purpose.
When we trade the cost of having these false failures against the cost of factor- ing out the file system, we tend to lean in the direction of the refactoring as soon as is feasible. Of course we do not want to sacrifice delivering business value to make the tests slightly better, but that depends on the number of tests. Five?
Twenty? Two hundred? The more file system tests there are, the higher the cost of leaving the dependency on the file system in place. As always, crunch the num- bers, and if they do not convince you, then leave the dependency in and log how much time you spend dealing with the problems that arise as a result. It is better to measure the impact than speculate about it.
608 CHAPTER 17 Odds and ends
◆ Related
■ 4.7—Control the order of some of your tests