Chuck Norris can unit test an entire application with a single assert.
Chapter 7. Points of Controversy
— Wisdom of the Internet ;)
The idea of having only One Assertion Per Test Method (I will use the acronym OAPTM for this from here on) is quite popular among developers. Some of its popularity may be attributed to the misuse of multi-assertion test methods, which tend to grow into many lines of code (in the hands of inexperienced or incompetent developers). It has also been discussed on popular forums and promoted by a number of prominent TDD and testing proponents, including Dave Astel and Roy Osherove.
All of the code examples presented so far in this book make it quite obvious that I myself do not subscribe to this idea and do not follow it. And even though I understand and share some of the motivations behind it, I will try to persuade you not to subscribe to it either – at least, not in its dogmatic version.
7.4.1. Code Example
Let us take a look at an example, to make sure we understand the problem in hand. I will use an example taken from the "One Assertion per Test" post by Dave Astels6. It relates to the problem of parsing some information passed as a String argument and creating a reasonable domain class out of it. In Listing 7.7 I use original test method names.
Listing 7.7. Address parsing – one assert per test method
public class AddressParsingOneAssertTest { private Address anAddress
= new Address("ADDR1$CITY IL 60563$COUNTRY");
@Test
public void testAddr1() {
assertEquals("ADDR1", anAddress.getAddr1());
} @Test
public void testCsp() {
assertEquals("CITY IL 60563", anAddress.getCsp());
} @Test
public void testCountry() {
assertEquals("COUNTRY", anAddress.getCountry());
} }
setting the test fixture for every test method, each test method contains only one assert.
As you can see, OAPTM in us having a lot of very focused, tiny, often even just one-line, test methods.
To get a broader understanding of "what address parsing looks like", you need to scan at least a few of them. What you get in return is that if a constructor of the Address class does not work properly, you
Chapter 7. Points of Controversy
Listing 7.8. Address parsing – several assertions per test method
public class AddressParsingManyAsserts { @Test
public void testAddressParsing() {
Address anAddress = new Address("ADDR1$CITY IL 60563$COUNTRY");
assertEquals("ADDR1", anAddress.getAddr1());
assertEquals("CITY IL 60563", anAddress.getCsp());
assertEquals("COUNTRY", anAddress.getCountry());
} }
Only one test method - with three assertions.
No need for setUp method.
This time the number of test methods is significantly smaller, but they themselves are slightly larger (though still reasonably short). It is very easy to understand what "address parsing" means, since all parsing assertions are grouped together. On the downside, though, it might happen that any bugs introduced into the Address class constructor only reveal themselves gradually – one by one, in consecutive runs of the AddressParsingManyAssertsTest test class.
A matcher-based solution would be even better. For
example: assertThat(anAddress).hasCountry("COUNTRY").hasCsp("CITY IL 60563").hasAddr1("ADDR1").
One important thing to notice is that even though in the second version of the test there is more than one assert per test method, each test method tests only one thing - the ability of the Address constructor to parse addresses properly.
7.4.2. Pros and Cons
Allow me now to take up the commonly cited reasons for following the OAPTM rule (as put forward by its proponents) and comment on them as they relate to the code present in the Listing 7.7 and Listing 7.8.
Table 7.2. Arguments for using only one assert per test
argument comment
Using more than one assertion per test method leads to testing more than one thing - which violates the SRP.
Use of many assertions per test method may potentially lead to testing too much, this is true. However, it does not have to be so. The question is, what granularity of this 'one thing' you are testing makes you feel comfortable. For me, testing the ability of Address class constructor to parse address passed as a String, is just the right size of a thing (of a functionality) to be tested in single test method.
Tests written this way are easier to maintain (thanks to smaller, focused methods).
This is a very subjective statement. The number of test methods created with OAPTM rule can be really high.
Maintaining them might be a nightmare, especially that you need to browse through many of them to get a clear picture of tested functionality.
Chapter 7. Points of Controversy
argument comment
When a test fails it is dead simply to deduce what went wrong.
If you write good test with many asserts you can achieve the same effect. Current IDEs are capable of telling you exactly which assertion failed, so I do not see any gain from using OAPTM in this aspect.
If more than one asserts are about to fail, they will - in case of multiple asserts per test, only the first one will fail, so you need to rerun your tests to see the next one failing.
This is true, but not something you should worry about.
From my experience, such situations (where one assertion failure "hides" another failures), is more than rare - in fact I can not recall it ever happened to me. I shield myself from such effect, writing assertions one by one, and following the TDD rhythm. Of course, this does not defend me against such effect, when I introduce some changes to an existing class. However, as stated previously, I believe this threat to be more of theoretical, than practical meaning.
It is easier to create intention-revealing method names for one-assertion tests.
Maybe so, but as proved by original code on Listing 7.7, it is still pretty easy to come with lousy names (e.g.
testCsp()) even if using one assert per test! I am confident that finding good intention-revealing names for many- assertions is possible, and that OATPM does not have any advantages here.
7.4.3. Conclusions
Even as a firm believer in small, focused, easy-to-understand methods, I think that OAPTM goes too far, and does more harm than good. It results in too many lines of code being scattered amongst a multitude of tiny methods – something which is much harder to understand and maintain. It saves me from one problem (having too many assertions per test method), only to inflict on me another (having too many test methods).
No. Sorry, OAPTM! I can judge on my own when a test method is getting too large, without needing to follow such dogmatic rules. And as long as I stick to the TDD approach, I will rarely, if ever, be stricken by the downside of the multiple asserts approach. My test methods are very concise and readable, and I can easily find intention-revealing names for them. My IDE will guide me straight to the fallen assertion every time a test fails.
My advice would be not to take the "one assert per test" recommendation too literally. Get clear about the reasons behind it, but follow another, similar rule, which says that each test method should concentrate on a single feature of the SUT. Sometimes numerous assertions will be required to test a certain feature, sometimes one will suffice. But in my opinion, the number of assertions should not be your main concern.
What really matters is the scope of the SUT features covered by each test method.
The only thing that would get me writing so many tiny test methods would be if I was being paid by LOCs. :)
Chapter 7. Points of Controversy