Chapter 6. Things You Should Know
6.9. Unit Testing Asynchronous Code
Chuck Norris can test multi-threaded applications with a single thread.
— Wisdom of the Internet ;)
Often we write code which delegates some task to be run "in the background", so the caller of our code will not have to wait for it to finish. This very popular scenario is usually handled with the
java.util.concurrent.ExecutorService14
.
I do not plan to go into details concerning writing asynchronous Java code. For the purpose of this discussion it will suffice to say that the implementation of the ExecutorService interface is capable of running tasks (Runnable and Callable) asynchronously. The submit() method of the ExecutorService
interface returns immediately, even if the execution of the task passed to it lasts for some time.
An example of such an approach is shown in Listing 6.22. A server receives a request and handles all the tasks of this request. The tasks are handled asynchronously.
14See http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/ExecutorService.html
Chapter 6. Things You Should Know
Listing 6.22. Starting tasks asynchronously with ExecutorService
public class Server {
private final ExecutorService executorService;
private final TaskService taskService;
public Server(ExecutorService executorService, TaskService taskService) { this.executorService = executorService;
this.taskService = taskService;
}
public void serve(Request request) { for (Task task : request.getTasks()) {
executorService.submit(new TaskHandler(taskService, task));
} }
private class TaskHandler implements Runnable { private final TaskService taskService;
private final Task task;
public TaskHandler(TaskService taskService, Task task) { this.taskService = taskService;
this.task = task;
}
public void run() { ...
taskService.handle(task);
} } }
Collaborators of the Server class are injected via a constructor.
The submit() method of the ExecutorService returns immediately, even if the execution of the
Runnable that was passed to it takes some time.
The TaskHandler class implements the Runnable interface.
The run() method is responsible for "doing the real work". We need not concern ourselves with what it does, but let us assume that its execution takes a significant amount of time.
Here the handle() method of the TaskService class is invoked, which finishes the execution of the run() method.
Now let us test the following scenario: We will send a request to the server and verify whether the
handle() method of the TaskService class has been executed.
Hmm…. It seems that to have real unit tests we should write tests for two classes:
• for the Server class to verify whether it submits all tasks to its executorService collaborator,
• and for the TaskHandler class to verify whether its run() method invokes the handle() method of the collaborator.
Chapter 6. Things You Should Know
Sounds simple, so let’s go ahead and do it, based on what we already know about mocks. The only question we should answer is what to do with ExecutorService. Should we use its real implementation or replace it with a test double? The first option seems better to me. First of all, the functionality we are testing is contained within two types (Server and ExecutorService), so mocking just one of them would be awkward. Secondly, there are many definitely valid implementations of the ExecutorService
interface provided by the mighty Java itself, so we need not worry about bugs within this collaborator of the ExecutorService class. Last but not least, we should stick to the good old rule which says "mock only types that you own"15: this tells us that since we don’t own the ExecutorService (in that we haven’t written it and have no control over its further development) we should not mock it. In sum, then, our decision is that we should use some real implementation of this interface.
Listing 6.23. Testing asynchronous code - a naive approach
@Test
public void shouldSaveTasks() throws InterruptedException {
ExecutorService executorService = Executors.newCachedThreadPool();
TaskService taskService = mock(TaskService.class);
Task task = mock(Task.class);
Collection<Task> listOfTasks = Arrays.asList(task);
Server server = new Server(executorService, taskService);
server.serve(listOfTasks);
verify(taskService).handle(task);
}
Collaborators of the SUT. As was previously decided, we mock an object of the TaskService class and use a real implementation of the ExecutorService interface.
The execution of the serve() method initiates the asynchronous task.
Verification occurs of whether the handle() method of the taskService collaborator is invoked at the end of the execution.
Will it work? Unfortunately not. At the time when Mockito runs the verification part of the test, the asynchronously invoked operations are still in progress, and the handle() method has not yet been invoked.
6.9.1. Waiting for the Asynchronous Task to Finish
It seems that we need to wait for the asynchronous task to finish. Let’s suppose that the first idea we come up with is to put a Thread.sleep() into our test code, like this:
Listing 6.24. Testing asynchronous code - with Thread.sleep()
@Test
public void shouldSaveTasks() throws InterruptedException { ...
server.serve(listOfTasks);
Thread.sleep(1000);
verify(taskService).handle(task);
}
15See http://stevef.truemesh.com/archives/000194.html
Chapter 6. Things You Should Know
Now the test is green and… slow! Every time we run it we lose 1 second. This does not seem to be a lot, but if you think about it you will come to the conclusion that it is a bad situation. What if we have more tests like this and the waiting time grows to 15 seconds, or maybe a minute? This would not be acceptable, as it would eventually lead to developers not running tests.
Of course, we could perform some fine-tuning and make the sleep shorter, but this is a risky business.
There is a risk that under some circumstances (on a different machine, or when garbage collection interferes) the waiting time will be too short, and the test will fail. False alarms are an awful thing – they make people indifferent to tests turning red!
Pursuing the direction we have already gone in (i.e. waiting till the asynchronous task is finished), we could do a little better if we repeatedly execute the verification until it succeeds, or until the execution time exceeds some reasonable value. This could be done with the try-catch statement, or with some fancy framework like Awaitility16. Both solutions are presented in the next two listings.
Listing 6.25. Testing asynchronous code - with Thread.sleep() within a for loop
@Test
public void shouldSaveTasksUsingTryCatch() throws InterruptedException { final TaskService taskService = mock(TaskService.class);
ExecutorService executorService = Executors.newCachedThreadPool();
final Task task = mock(Task.class);
Collection<Task> listOfTasks = Arrays.asList(task);
Server server = new Server(executorService, taskService);
server.serve(listOfTasks);
boolean handleMethodInvoked = false;
for (int i = 0; i < 10; i++) { try {
verify(taskService).handle(task);
handleMethodInvoked = true;
}
catch (AssertionError e) { // no need to do anything }
Thread.sleep(100);
}
assertTrue(handleMethodInvoked);
}
This flag will allow us to verify whether the expected handle() method has really been executed.
The maximum waiting time is 1 second (100 milliseconds multiplied by 10).
If the verification fails, then an AssertionError will be thrown. There is nothing we need do within the catch clause, because we will want to try again after some time.
If we get there, it means the verification has succeeded.
Let us see whether, during the (maximum) time of 1 second, the expected method was invoked.
Is it better than the previous version? Yes, because it is guaranteed that we will wait, at the very most, 100 milliseconds longer after the task finished, and at the same time we will never wait longer than 1 second. However, the code gets complicated. To enhance its readability, we could extract the for loop
Chapter 6. Things You Should Know
Listing 6.26. Testing asynchronous code - with Awaitility
Awaitility.await() .atMost(1, SECONDS)
.with().pollInterval(100, MILLISECONDS) .until(new Callable<Boolean>() {
@Override
public Boolean call() throws Exception { try {
verify(taskService).handle(task);
return true;
} catch(AssertionError ae) { return false;
} } });
Personally I do not like the try-catch version of this test. It is much more complicated than the usual tests. The second version - which uses Awaitility - is more readable, but even so, it also requires us to catch a class of the Error type, which goes against the common rules of Java coding.
Spend some time learning about the Awaitility framework. You won’t use it a lot in unit tests, but it can do wonders to your test code in your integration or end-to-end tests, making it much more readable.
6.9.2. Making Asynchronous Synchronous
So far we have been concentrating on waiting for the asynchronous code. This is possible, but not without some drawbacks. Let us try a different approach now.
Let us take a look again at the code in Listing 6.22. Can you see anything which makes it asynchronous?
No, there is nothing of that sort there. The asynchronous nature of this code is "injected" by a concrete implementation of the ExecutorService, which means, in turn, that if we provide a synchronous implementation of this interface, it will make our situation much simpler. At the same time, by doing this we won’t breach the contract of the ExecutorService. Even if it was designed with asynchronicity in mind, it does not demand implementations to provide such a feature.
Listing 6.27 presents such a sample implementation.
Chapter 6. Things You Should Know
Listing 6.27. SynchronousExecutorService
public class SynchronousExecutorService extends AbstractExecutorService { private boolean shutdown;
public void shutdown() { shutdown = true;
}
public List<Runnable> shutdownNow() { shutdown = true;
return Collections.emptyList();
}
public boolean isShutdown() { shutdown = true;
return shutdown;
}
public boolean isTerminated() { return shutdown;
}
public boolean awaitTermination(final long timeout, final TimeUnit unit) {
return true;
}
public void execute(final Runnable command) { command.run();
} }
This implementation extends the java.util.concurrent.AbstractExecutorService class, which allows us to implement only selected methods of the ExecutorService interface.
This is the most important part of this class: instead of executing the command in a separate thread we simply run it synchronously.
If we inject this synchronous implementation into the tested instance of the Server class, we discover that the test is now trivial. By removing the asynchronicity we have simplified this case, turning it into a normal one. There is no need for any waiting within the test code.
Listing 6.28. Testing asynchronous code - using SynchronousExecutorService
@Test
public void shouldSaveTasks() {
TaskService taskService = mock(TaskService.class);
ExecutorService executorService = new SynchronousExecutorService();
Server server = new Server(executorService, taskService);
Task task = mock(Task.class);
Collection<Task> listOfTasks = Arrays.asList(task);
server.serve(listOfTasks);
verify(taskService).handle(task);
}
Chapter 6. Things You Should Know
thing to happen. The second option is to reduce the asynchronous code to its synchronous counterpart and then solve it using a standard approach.
If you are interested in testing asynchronous and concurrent code then [goetz2006] is probably the book for you to read.