Chapter 4 covered all Spock blocks in detail as well as the general structure of a Spock test. Spock offers several complementary features in the form of annotations that fur- ther enhance the expressiveness of your tests.
The Spock tests demonstrated here are based on the e-shop example introduced in chapter 6. They revolve around placing products in an electronic basket and paying via credit card.
8.1.1 Testing the (non)existence of exceptions: thrown() and notThrown()
In all Spock tests that I’ve shown you so far, the expected result is either a set of asser- tions or the verification of object interactions. But in some cases, the “expected” result is throwing an exception. If you’re developing a library framework, for example, you have to decide what exceptions will be thrown to the calling code and verify this deci- sion with a Spock test. The next listing demonstrates the capturing of an exception.
def "Error conditions for unknown products"() { given: "a warehouse"
WarehouseInventory inventory = new WarehouseInventory() when: "warehouse is queried for the wrong product"
inventory.isProductAvailable("productThatDoesNotExist",1) then: "an exception should be thrown"
thrown(IllegalArgumentException) }
1 Mockito’s official documentation also has a huge warning against the usage of spies.
Listing 8.1 Expecting an exception in a Spock test
This test will pass only if IllegalArgumentException is thrown in the when: block.
Here you design the Warehouse class so that it throws an exception when it can’t find the name of a product. When this test runs, you explicitly tell Spock that the when:
block will throw an exception. The test will fail if an exception isn’t thrown, as shown in figure 8.1.
In this case, you use an existing exception as offered by Java, but the same syntax works with any kind of exception. (You could create a custom exception class called ProductNotFoundException instead.)
It’s also possible to “capture” the exception thrown and perform further assertions in order to make the test stricter. The following listing provides an example of a mes- sage of an exception that’s checked.
def "Error conditions for unknown products - better"() { given: "a warehouse"
WarehouseInventory inventory = new WarehouseInventory() when: "warehouse is queried for the wrong product"
inventory.isProductAvailable("productThatDoesNotExist",1) then: "an exception should be thrown"
IllegalArgumentException e = thrown() e.getMessage() == "Unknown product productThatDoesNotExist"
}
This listing further enhances the code of listing 8.1 by checking both the type of the exception and its message. Here you examine the built-in message property that’s pres- ent in all Java exceptions, but again, you could examine any property of a custom-made exception instead (the last statement in listing 8.2 is a standard Groovy assertion).
Finally, it’s possible to define in a Spock test that you don’t expect an exception for an operation in the when: block, as the following listing shows. I admit that the seman- tics of this syntax are subtle, but the capability is there if you need it.
Listing 8.2 Detailed examination of an expected exception Figure 8.1 The test will fail if an exception isn’t thrown in the when: block.
Keeps the exception thrown in the e variable The test will
pass only if the exception contains a specific message.
227 Using additional Spock features for enterprise tests
def "Negative quantity is the same as 0"() { given: "a warehouse"
WarehouseInventory inventory = new WarehouseInventory() and: "a product"
Product tv = new Product(name:"bravia",price:1200,weight:18) when: "warehouse is loaded with a negative value"
inventory.preload(tv,-5)
then: "the stock is empty for that product"
notThrown(IllegalArgumentException) !inventory.isProductAvailable(tv.getName(),1) }
I believe that the notThrown() syntax is intended as a hint to the human reader of the test and not so much to the test framework itself.
8.1.2 Mapping Spock tests to your issue-tracking system: @Issue
In chapter 4, you saw the @Subject, @Title, and @Narrative annotations that serve as metadata for the Spock test. These annotations are particularly useful to nontechnical readers of the tests (for example, business analysts) and will show their value when reporting tools use them for extra documentation.
Any nontrivial enterprise application has a product backlog or issue tracker that serves as a central database of current bugs and future features. Spock comes with an
@Issue annotation that allows you to mark a test method that solves a specific issue with the code, as shown in the following listing.
@Issue("JIRA-561") def "Error conditions for unknown products"() {
given: "a warehouse"
WarehouseInventory inventory = new WarehouseInventory() when: "warehouse is queried for the wrong product"
inventory.isProductAvailable("productThatDoesNotExist",1) then: "an exception should be thrown"
thrown(IllegalArgumentException) }
Notice that the annotation has a strictly informational role. At least at the time of writ- ing, no automatic connection to any external system exists (in this example, to JIRA, available at www.atlassian.com/software/jira). In fact, the value inside the annotation is regarded as free text by Spock. The next listing shows another example using a full URL of a Redmine tracker (www.redmine.org).
Listing 8.3 Explicit declaration that an exception shouldn’t happen
Listing 8.4 Marking a test method with the issue it solves
Clarifies the intention of testing normal operation without exception
This test method verifies the fix that happened for JIRA issue 561.
@Issue("http://redmine.example.com/issues/2554") def "Error conditions for unknown products - better"() { given: "a warehouse"
WarehouseInventory inventory = new WarehouseInventory() when: "warehouse is queried for the wrong product"
inventory.isProductAvailable("productThatDoesNotExist",1) then: "an exception should be thrown"
IllegalArgumentException e = thrown()
e.getMessage() == "Uknown product productThatDoesNotExist"
}
Finally, a common scenario is having multiple issue reports that stem from the same problem. Spock has you covered, and you can use multiple issues, as shown in the fol- lowing listing.
@Issue(["JIRA-453","JIRA-678","JIRA-3485"]) def "Negative quantity is the same as 0"() {
given: "a warehouse"
WarehouseInventory inventory = new WarehouseInventory() and: "a product"
Product tv = new Product(name:"bravia",price:1200,weight:18) when: "warehouse is loaded with a negative value"
inventory.preload(tv,-5)
then: "the stock is empty for that product"
notThrown(IllegalArgumentException)
!inventory.isProductAvailable(tv.getName(),1) }
The @Issue annotation is also handy when you practice test-driven development, as you can use it to mark Spock tests for product features before writing the production code.
8.1.3 Failing tests that don’t finish on time: @Timeout
Chapter 7 covered integration and functional tests and how they differ from pure unit tests. A common characteristic of integration tests is their slow execution time because of real databases, web services, and external systems that are often used as part of the test.
Getting quick feedback from a failed unit test should be one of your primary goals when writing integration tests. The external systems used in integration tests can affect the execution time in a nondeterministic way, as their response time is affected by their current load or other environmental reasons.
Listing 8.5 Using the URL of an issue solved by a Spock test
Listing 8.6 Marking a Spock test with multiple issues
This test method verifies the fix that happened for Redmine issue 2554.
This test method verifies the fix for three duplicate bugs.
229 Using additional Spock features for enterprise tests
Spock comes with an @Timeout annotation that unconditionally fails a test if its execution time passes the given threshold. The following listing shows an example.
@Timeout(5) def "credit card charge happy path"() {
given: "a basket, a customer and a TV"
Product tv = new Product(name:"bravia",price:1200,weight:18) BillableBasket basket = new BillableBasket()
Customer customer = new
Customer(name:"John",vip:false,creditCard:"testCard") and: "a credit card service"
CreditCardProcessor creditCardSevice = new CreditCardProcessor() basket.setCreditCardProcessor(creditCardSevice)
when: "user checks out the tv"
basket.addProduct tv
boolean success = basket.checkout(customer) then: "credit card is charged"
success }
The reasoning behind the @Timeout annotation is that it helps you quickly isolate environmental problems in your integration tests. If a service is down, there’s no point in waiting for the full time-out of your Java code (which could be 30 minutes, for example) before moving to the next unit test.
Using the @Timeout annotation, you can set your own bounds on the “expected”
runtime of an integration test and have Spock automatically enforce it. The default unit is seconds, as shown in the previous listing, but you can override it with your own setting, as shown in the next listing.
@Timeout(value = 5000, unit = TimeUnit.MILLISECONDS) def "credit card charge happy path - alt "() {
given: "a basket, a customer and a TV"
Product tv = new Product(name:"bravia",price:1200,weight:18) BillableBasket basket = new BillableBasket()
Customer customer = new
Customer(name:"John",vip:false,creditCard:"testCard") and: "a credit card service"
CreditCardProcessor creditCardSevice = new CreditCardProcessor() basket.setCreditCardProcessor(creditCardSevice)
when: "user checks out the tv"
basket.addProduct tv
boolean success = basket.checkout(customer) then: "credit card is charged"
success }
Listing 8.7 Declaring a test time-out
Listing 8.8 Declaring a test time-out—custom unit
This test should finish within five seconds.
The Credit- CardProcessor class is an external service.
This is a lengthy operation that contacts the credit card service.
Treats the defined value as milliseconds
The importance of the @Timeout annotation is evident in the case of multiple long tests that take a long time to finish. I’ve seen build jobs that typically take minutes but because of a misconfiguration can take hours if time-outs aren’t used correctly.
8.1.4 Ignoring certain Spock tests
A large enterprise application can have thousands of unit tests. In an ideal world, all of them would be active at any given time. In real life, this is rarely the case.
Test environments that get migrated, features that wait to be implemented, and business requirements that aren’t yet frozen are common reasons that force some tests to be skipped. Fortunately, Spock offers several ways to skip one or more tests deliberately so your tests don’t fail while these restructurings and developments are taking place.
IGNORING A SINGLE TEST: @IGNORE
Spock allows you to knowingly skip one or more tests and even provides you with the ability to give a reason for skipping that test (see the next listing).
@Ignore("Until credit card server is migrated") def "credit card charge happy path"() {
given: "a basket, a customer and a TV"
Product tv = new Product(name:"bravia",price:1200,weight:18) BillableBasket basket = new BillableBasket()
Customer customer = new
Customer(name:"John",vip:false,creditCard:"testCard") and: "a credit card service"
CreditCardProcessor creditCardSevice = new CreditCardProcessor() basket.setCreditCardProcessor(creditCardSevice)
when: "user checks out the tv"
basket.addProduct tv
boolean success = basket.checkout(customer) then: "credit card is charged"
success }
The primary purpose of skipping a test is so that the rest of your test suite is run suc- cessfully by your build server. An ignored test should always be a temporary situation because you’re vulnerable to code changes that would normally expose a bug verified by that test.
The human-readable description inside the @Ignore annotation should give a hint about why this test is ignored (the value is free text, as far as Spock is concerned).
More often than not, the original developer who marks a test as ignored doesn’t always remove the @Ignore annotation, so it’s essential to document inside the source code the reason why the test was skipped in the first place.
Listing 8.9 Ignoring a single test
This test will be skipped when Spock runs it.
231 Using additional Spock features for enterprise tests
You can place @Ignore on a single test method or on a whole class if you want all its test methods to be skipped.
IGNORING ALL BUT ONE TEST: @IGNOREREST
If you’re also lucky, and you want to ignore all but one test in a Spock specification, you can use the @IgnoreRest annotation. Assume that you have a set of integration tests that contact a credit card external service in a staging environment (it doesn’t actually charge cards). The service is down for maintenance. To keep your tests run- ning, you could ignore tests selectively, as shown in the following listing.
class KeepOneSpec extends spock.lang.Specification{
def "credit card charge - integration test"() { [...code redacted for brevity...]
}
@IgnoreRest def "credit card charge with mock"() { [...code redacted for brevity...]
}
def "credit card charge no charge - integration test"() { [...code redacted for brevity...]
} }
Running the Spock test shown in this listing produces the output in figure 8.2.
Again, I admit that this Spock annotation is specialized, and you might never need to use it.
IGNORING SPOCK TESTS ACCORDING TO THE RUNTIME ENVIRONMENT: @IGNOREIF PART 1
The @Ignore annotations shown in the previous paragraph are completely static. A test is either skipped or not, and that decision is made during compile time.
Listing 8.10 Ignoring all tests except one
This test uses the real credit card service—it will be skipped.
Marking this test as the only one that will run
This test uses only mocks and thus can run normally
This test uses the real credit card service—it will be skipped.
Figure 8.2 Only the test marked with @IgnoreRest runs.
Spock offers a set of smarter @Ignore annotations that allow you to skip tests dynami- cally (by examining the runtime environment). As a first step, Spock allows a test to query the following:
■ The current environment variables
■ The JVM system properties
■ The operating system
Spock then decides whether the test will run, depending on that result. An example of skipping tests is shown in the next listing.
class SimpleConditionalSpec extends spock.lang.Specification{
@IgnoreIf({ jvm.java9 }) def "credit card charge happy path"() {
[...code redacted for brevity...]
}
@IgnoreIf({ os.windows }) def "credit card charge happy path - alt"() {
[...code redacted for brevity...]
}
@IgnoreIf({ env.containsKey("SKIP_SPOCK_TESTS") }) def "credit card charge happy path - alt 2"() {
[...code redacted for brevity...]
} }
Running this listing on my Windows system with JDK 7 and no extra JVM properties produces the output shown in figure 8.3.
Listing 8.11 Skipping Spock tests according to the environment
This test will be skipped on Java 9.
This test will be skipped if run on Windows.
This test will be skipped if the property SKIP_SPOCK_TESTS is defined.
Figure 8.3 A test is skipped because the current OS is Windows.
233 Using additional Spock features for enterprise tests
I won’t list all possible options supported by Spock. You can find the full details in its source code.2 Ignoring tests depending on environment variables enables you to split your tests into separate categories/groups, which is a well-known technique. As an example, you could create “fast” and “slow” tests and set up your build server with two jobs for different feedback lifecycles.
IGNORING CERTAIN SPOCK TESTS WITH PRECONDITIONS: @IGNOREIF PART 2
To obtain the maximum possible flexibility from @IgnoreIf annotations, you need to define your own custom conditions. You can do this easily in Spock because the
@IgnoreIf annotation accepts a full closure. The closure will be evaluated and the test will be skipped if the result is false.
The following listing shows a smarter Spock test that runs only if the CreditCard- Service is up and running.
@IgnoreIf({ !new CreditCardProcessor().online() }) def "credit card charge happy path - alt"() {
given: "a basket, a customer and a TV"
Product tv = new Product(name:"bravia",price:1200,weight:18) BillableBasket basket = new BillableBasket()
Customer customer = new
Customer(name:"John",vip:false,creditCard:"testCard") and: "a credit card service"
CreditCardProcessor creditCardSevice = new CreditCardProcessor() basket.setCreditCardProcessor(creditCardSevice)
when: "user checks out the tv"
basket.addProduct tv
boolean success = basket.checkout(customer) then: "credit card is charged"
success }
This listing assumes that the Java class representing the external credit card system has a built-in method called online() that performs a “ping” on the remote host. Spock runs this method, and if it gets a negative result, it skips the test (there’s no point in running it if the service is down).
The contents of the closure passed as an argument in the @IgnoreIf annotation can be any custom code you write. If, for example, the built-in online() method wasn’t present, you could create your own Java (or Groovy) class that performs an HTTP request (or something appropriate) to the external system and have that inside the closure.
2 https://github.com/spockframework/spock/tree/master/spock-core/src/main/java/spock/util/
environment
Listing 8.12 Skipping a Spock test based on a dynamic precondition
This test will run only if the method online() of the credit card service returns true.
The Credit- CardProcessor class is an external service.
This operation contacts the credit card service.
REVERSING THE BOOLEAN CONDITION OF IGNOREIF: @REQUIRES
If for some reason you find yourself always reverting the condition inside the
@IgnoreIf annotation (as seen in listing 8.12, for example), you can instead use the
@Requires annotation, as the following listing shows.
@Requires({ new CreditCardProcessor().online() }) def "credit card charge happy path"() {
given: "a basket, a customer and a TV"
Product tv = new Product(name:"bravia",price:1200,weight:18) BillableBasket basket = new BillableBasket()
Customer customer = new
Customer(name:"John",vip:false,creditCard:"testCard") and: "a credit card service"
CreditCardProcessor creditCardSevice = new CreditCardProcessor() basket.setCreditCardProcessor(creditCardSevice)
when: "user checks out the tv"
basket.addProduct tv
boolean success = basket.checkout(customer) then: "credit card is charged"
success }
The @Requires annotation has the same semantics as @IgnoreIf but with the reverse behavior. The test will be skipped by Spock if the code inside the closure does not eval- uate to true. The option to use one or the other annotation comes as a personal pref- erence.
8.1.5 Automatic cleaning of resources: @AutoCleanup
Chapter 4 showed you the cleanup: block as a way to release resources (for example, database connections) at the end of a Spock test regardless of its result. An alternative way to achieve the same thing is by using the @AutoCleanup annotation, as shown in the following listing.
@AutoCleanup("shutdown") private CreditCardProcessor creditCardSevice = new CreditCardProcessor() def "credit card connection is closed down in the end"() {
given: "a basket, a customer and a TV"
Product tv = new Product(name:"bravia",price:1200,weight:18) BillableBasket basket = new BillableBasket()
Customer customer = new
Customer(name:"John",vip:false,creditCard:"testCard") Listing 8.13 Requires is the opposite of IgnoreIf
Listing 8.14 Releasing resources with AutoCleanup
This test will run only if the method online() of the credit card service returns true.
The Credit- CardProcessor class is an external service.
This operation contacts the credit card service.
The shutdown() method of the credit card service will be called at the end of the tests.
235 Handling large Spock tests
and: "a credit card service"
basket.setCreditCardProcessor(creditCardSevice) when: "user checks out the tv"
basket.addProduct tv
boolean success = basket.checkout(customer) then: "credit card is charged"
success }
If you mark a resource with the @AutoCleanup annotation, Spock makes sure that the close() method will be called on that resource at the end of the test (even if the test fails). You can use the annotation on anything you consider a resource in your tests.
Database connections, file handles, and external services are good candidates for the
@AutoCleanup annotation.
You can override the method name that will be called by using it as an argument in the annotation, as done in listing 8.11. In that example, the shutdown() method will be called instead (Spock will call close() by default).
I prefer to use the cleanup: block and cleanup()/cleanupSpec() methods as explained in chapter 4 (especially when multiple resources must be released), but if you’re a big fan of annotations, feel free to use @AutoCleanup instead.3 As you might guess, @AutoCleanup works both with instance fields and objects marked with the
@Shared annotation shown in chapter 4.
This concludes the additional Spock annotations,4 and we can now move to refac- toring of big Spock tests.