Equivalence partitioning is a well-known technique for separating values of a test input into classes of equivalent expected results. With bound- ary value analysis, a small number of values can be selected from each partition to exercise the SUT for the corresponding result, and to check that the partition boundaries are implemented correctly. Sections 4.1 and 5.6 described these methods as applied to input values. In this chapter, a behavior refers to a class of equivalent results involving transition(s) among diagrammed states. The specific results comprising the equivalence class reflect the program variables of the extended state as well. Thus, each par- tition has an expected behavior, and the partition’s test cases initiate spe- cific expected results.
An application for Fahrenheit to Celsius temperature conversion pro- vides a simple example of partitioning. For inputs above absolute zero, the application is required to convert the temperature according to the usual formula; otherwise, an exception response is given. These two behaviors correspond to two partitions, and a small number of input values can be selected from each.
In more complex applications, there may be more classes of equivalent results. Additional partitions are needed then: Different normal behaviors require different partitions. And if different types of exception process- ing are to be verified individually, they need separate partitions as well.
Moreover, the presence of multiple inputs implies partitions with multiple dimensions—one for each input—as in the following example.
6.1.1 Partitions for Expected Results
Figure 6.1 illustrates a design-your-burger application using an HTML form [181]. Inputs include a burger menu, a cooking choice radio button, eight topping checkboxes, buttons to reset or submit the form, and links to other pages. The purpose of the test is to verify the action of the form with all pairs of input values. A reasonable starting point is to associate each input with a test factor. However, four of the inputs—the reset, submit, and link inputs—are different from the others because they change the state.
The submit button is the trigger to start the processing of the form’s data.
FIGURE 6.1 Design-your-burger form. Inputs include a burger menu (beef, tur- key, or veggie), a cooking choice radio button, eight topping checkboxes, buttons to reset or submit the form and links to other pages.
116 ◾ Introduction to Combinatorial Testing
The reset button clears the form, and the links go to the Nutrition facts page or the About the chef page. This partition is to verify the behavior of the submit button and the resulting processing. Inputs from the reset but- ton and the links do not lead to the expected result state and should not be included in the partition. If all the inputs are put into the same partition, we can expect some input value pairs will not test the action of the form.
Instead they will be reset or lost on the way to other pages.
Table 6.1 shows the 10 test factors chosen for this partition—the menu, radio button, and eight checkboxes—and their test cases. The submit but- ton is clicked after the other input values are set. Each pair of input values is covered.
6.1.2 Partitions with Constraints
Typically test inputs need constraints so that the combinations of values in a partition lead to equivalent expected results (one behavior). If an inter- face takes months, days, and years as inputs for valid dates, day 31 should give a normal result only for the long months, January, March, May, and so on. If April 31 is a possible input, it should lead to an exception in a differ- ent partition instead. April 31 is an example of an invalid combination as described in Section 3.3.2. However, the reason that April 31 requires a dif- ferent partition from April 30 is that a different behavior is expected from the SUT. When two valid inputs are expected to cause different behaviors, they need to be in separate partitions as well. Generally, input constraints are needed to ensure that each partition has a set of test cases that covers its equivalence class and does not encroach into another partition.
Combinations that belong in a partition are allowed by the partition;
those that do not are disallowed. A partition with valid dates allows April 30 but not April 31. A partition with invalid dates that have too many days in a month allows April 31 but not April 30 (e.g., for an exception).
6.1.3 Direct Product Block Notation
The examples in this chapter use direct product block (DPB) notation [197] to specify test factor values and their constraints. In DPB notation each factor’s allowed values appear on a separate line in a block, and all combinations of values in the block are allowed. Partitions consist of one or more blocks. A partition’s allowed combinations are the union of its blocks’ combinations. Importantly, the blocks are defined so that they do not include any disallowed combinations. Table 6.2 gives an example of DPB notation to specify valid dates according to calendar rules. All
TABLE 6.1Design-Your-Burger Form Inputs BurgerCookedCheeseLettuceTomatoOnionKetchupMustardMayoSecret Sauce TurkeyWellYesYesNoYesNoNoNoYes VeggieNoneYesNoYesNoYesYesYesNo BeefRareNoYesNoYesYesNoYesNo TurkeyMediumNoNoYesYesYesYesYesYes BeefNoneNoNoNoNoNoNoNoYes VeggieRareYesNoYesNoNoYesNoYes VeggieMediumYesYesNoNoNoNoYesNo BeefWellNoNoYesNoYesYesYesYes TurkeyRareNoYesYesNoYesNoNoNo TurkeyNoneYesYesNoYesNoYesNoNo VeggieWellNoYesNoYesYesNoYesNo BeefMediumYesYesYesYesNoNoNoYes Source: Adapted from Sherwood, G. B. Getting the Most from Pairwise Testing: A Guide for Practicing Software Engineers. CreateSpace, Scotts Valley, California, 2011. Note: This pairwise design tests the action of the submit button in Figure 6.1. All pairs of test inputs are covered in these 12 test cases. Inputs for the reset button and the links to other pages are excluded from this partition because they change the state without leading to the expected result.
118 ◾ Introduction to Combinatorial Testing
combinations from each block are allowed, and the set of all allowed com- binations conforms to the partition boundaries.
DPB notation is independent of the strength of the design. That is, the same DPB partition defines the allowed t-tuples for any t up to the total number of factors. There can be various ways to choose the blocks in a partition. DPB notation allows blocks that are disjoint or overlapping in various ways. These block definitions are equivalent when they specify the same combinations. A typical approach for choosing a partition’s blocks is to select a block that covers most or all of the partition, and then split and/
or add blocks to account for dependencies and boundaries. The calendar example in Table 6.2 illustrates this approach as well.
TABLE 6.2 Partition for Valid Calendar Dates in DPB Notation Calendar Example
Month Day Year
# valid dates partition
+ all months, first & tenth days
jan feb mar apr may jun jul aug sep oct nov dec 1 10
2015 2016 2017
+ long month last day jan mar may jul aug oct dec 31
2015 2016 2017
+ short month last day apr jun sep nov
30
2015 2016 2017
+ feb last day, common year feb
28
2015 2017
+ feb last day, leap year feb
29 2016
Source: Courtesy of Testcover.com. www.testcover.com.
Note: The first four lines give names for the request and the three test factors. The next line, starting with the pound (#), starts the partition for valid dates. The partition boundar- ies conform to calendar rules using five blocks, indicated by lines starting with plus (+) and a comment. The first block includes all combinations of months and selected years, with first and tenth days of the month. The second and third blocks split months according to length and use the corresponding last day (31 or 30). The fourth and fifth blocks split years into common and leap years for February’s last day (28 or 29).