The sequence unit replay model applies combinatorial design to test a sequence of state transitions with comparisons to expected results. Each partition corresponds to a state transition path such that equivalent behavior is expected at each transition. Combinations of parameters are selected to replay different runs. The partition’s test cases all follow the same path and cover all allowed t-tuples. The state transition paths are chosen to meet test goals.
Each sequence unit consists of test inputs (e.g., from one or more test automation scripts), which are run as a contiguous unit during test exe- cution. Sequence units may have parameters to specify which inputs are exercised, and with which values. Examples of sequence units for an online shopping cart user interface (Figure 6.2) are listed in Table 6.5.
The test factors are sequence unit replay slots, ordered chronologically, to exercise the system behavior of interest. Thus, earlier sequence units not only test the system; they also set up the system for later interactions.
FIGURE 6.2 Shopping cart containing three different items. Items can be checked for deletion, and new quantities can be entered. The update button initi- ates these changes; the shop button returns to the previous shopping page; the checkout button presents the user interface for payment and shipping. An empty cart does not provide buttons for update or checkout.
Generally, test inputs and result checks may use multiple interfaces, as long as the timing of interactions is coordinated to be repeatable. Table 6.6 shows the test factors and values for the shopping cart example in DPB notation. In this partition, the slots correspond to placing different items into the cart, possibly checking them for deletion or entering new quan- tities, and possibly updating the cart and proceeding to checkout. Each block is constrained not to have an empty cart before update and before checkout, because empty carts cannot be updated or proceed to checkout.
Table 6.7 illustrates test cases generated for this shopping cart sequence unit replay design. All allowed pairs of the selected test factors are covered.
The success of a sequence unit replay design depends on each test case of a partition taking the SUT through a sequence of equivalent state tran- sitions (one path, one behavior). This is a necessary condition to cover the allowed combinations of inputs. Otherwise some combinations may not be used as intended, as in the design-your-burger example. State transition paths may be selected in different ways to meet this requirement. If a state machine diagram is available, the state transition paths can be chosen sys- tematically, using known state machine test methods [16].
Because the sequence unit replay model does not dictate the transition paths, it offers flexibility. It can be applied to selected SUT behaviors to support goals like the following.
TABLE 6.5 Shopping Cart Sequence Units
Sequence Unit Replay Action
suShopA Go to shopping page for itemA and add it to the shopping cart. Go to the cart.
suShopB Go to shopping page for itemB and add it to the shopping cart. Go to the cart.
suShopC Go to shopping page for itemC and add it to the shopping cart. Go to the cart.
suCart Go to the cart.
suDelete(i) Check (or uncheck) the cart delete box for the item in position i.
suNewQ(i,q) Type q in the cart quantity box for the item in position i.
suUpdate Click the cart update button.
suCheckout Click the cart checkout button.
Source: Courtesy of Testcover.com. www.testcover.com.
Note: Test automation sequence units and their parameters are the factor values in a sequence unit replay model. These sequence units are the basis of the shopping cart test design.
124 ◾ Introduction to Combinatorial Testing
TABLE 6.6Shopping Cart Test Partition for Sequence Unit Replay Model Shopping Cart Example - sequence unit replay model+ n= 2, no suDelete(1), no suNewQ(1,0) Position0suShopA suShopB Position1suShopC Position2n/a Delete0n/a suDelete(0) Delete1n/a Delete2n/a NewQ0n/a suNewQ(0,0) suNewQ(0,1) suNewQ(0,2) suNewQ(0,10) NewQ1n/a suNewQ(1,1) suNewQ(1,2) suNewQ(1,10) NewQ2n/a Updaten/a suUpdate Checkoutn/a suCheckout #+ n= 3, no suDelete(2), no suNewQ(2,0) + n= 1, no suDelete(0), no suNewQ(0,0)suShopA suShopA suShopB suShopCsuShopB n/asuShopC n/an/a suDelete(0) n/an/a suDelete(1) n/an/a n/an/a suNewQ(0,0) suNewQ(0,1) suNewQ(0,2) suNewQ(0,10) n/a suNewQ(0,1) su NewQ(0,2) suNewQ(0,10)n/a suNewQ(1,0) suNewQ(1,1) suNewQ(1,2) suNewQ(1,10) n/an/a suNewQ(2,1) suNewQ(2,2) suNewQ(2,10) n/an/a suUpdate n/a suUpdaten/a suCheckout n/a suCheckout Note: Test factors correspond to sequence unit replay slots. One to three different items are placed into the cart in the first three slots. Items are checked for deletion and have new quantities entered in the next six slots. The cart may be updated and checked out in the last two slots. The three blocks handle the cases of one, two, and three initial items, respectively. Each block is constrained not to have an empty cart before update and before checkout. A factor with the value n/a indicates that a sequence unit is not run in that slot.
TABLE 6.7Test Cases for Shopping Cart Sequence Unit Replay Model Position0Position1Position2Delete0Delete1Delete2NewQ0NewQ1NewQ2UpdateCheckout suShopBn/an/an/an/an/asuNewQ(0,10)n/an/asuUpdatesuCheckout suShopAsuShopBsuShopCsuDelete(0)suDelete(1)n/asuNewQ(0,1)suNewQ(1,10)suNewQ(2,1)n/an/a suShopAsuShopCn/asuDelete(0)n/an/asuNewQ(0,0)suNewQ(1,1)n/an/asuCheckout suShopAsuShopBsuShopCn/asuDelete(1)n/an/asuNewQ(1,2)suNewQ(2,10)suUpdaten/a suShopAsuShopBsuShopCsuDelete(0)n/an/asuNewQ(0,0)suNewQ(1,0)suNewQ(2,2)suUpdaten/a suShopBsuShopCn/an/an/an/asuNewQ(0,2)suNewQ(1,2)n/an/an/a suShopCn/an/an/an/an/asuNewQ(0,1)n/an/asuUpdaten/a suShopAsuShopBsuShopCsuDelete(0)suDelete(1)n/asuNewQ(0,10)n/asuNewQ(2,10)suUpdatesuCheckout suShopAsuShopCn/asuDelete(0)n/an/an/asuNewQ(1,10)n/asuUpdatesuCheckout suShopAsuShopBsuShopCsuDelete(0)suDelete(1)n/asuNewQ(0,2)suNewQ(1,2)suNewQ(2,1)suUpdatesuCheckout Note:The first 10 of the 35 shopping cart test cases are shown. All test cases start with an empty cart. In the first test case itemB is placed in cart position0; the quantity of this item is set to 10; the cart is updated and then sent to checkout.
Free ebooks ==> www.Ebook777.com
126 ◾ Introduction to Combinatorial Testing
• Test the delivered functions of an incomplete system.
• Test a system including externally provided components not specified in detail.
• Test a system for which operational specifications are not complete or up to date.
• Focus testing on behaviors of particular importance.
To apply the sequence unit replay model in these situations, test design- ers may need to construct or modify a state machine diagram so that the test design conforms to applicable requirements.
The choice of test model needs to reflect appropriate tradeoffs among project goals, resources, and risks. The flexibility of transition path selec- tion in this model comes with a risk of omission of some transitions. The test models presented in the rest of this chapter include all transitions to avoid this risk. For comparison, the next section presents a combinatorial test design for the same online shopping cart.