A Non-Deterministic T-Way Strategy with Seeding and Constraints Support
2012 IEEE Symposium on Humanities, Science and Engineering Research A Non-Deterministic T-Way Strategy with Seeding and Constraints Support Juliana Md Sharif1, Kamal Z Zamli2, Amir Abu Bakar3, Samihah Abdullah1, Iza Sazanita Isa1, Ida Rahayu Mohamed Noordin1 Faculty of Electrical Engineering, Universiti Teknologi MARA MALAYSIA, School of Electrical and Electronic Engineering, Universiti Sains MALAYSIA, 3Jabatan Kejuruteraan Elektrik, Politeknik Seberang Perai MALAYSIA {juliana394, samihah.abdullah,izasazanita,idarahayu660}@ppinang.uitm.edu.my, 2eekamal@eng.usm.my, 3ametlah@gmail.com Abstract— T-way strategy has been known to be effective in generating test data and addressed combinatorial explosion issue This paper describes the design and development of a nondeterministic t-way strategy, called TTG, with seeding and constraints support In doing so, a number of experiments are discussed in order to prove the correctness of the implementation Keywords- T-Way strategy; Constraint and Seeding support; Software and hardware testing I to be as part of the final suite (e.g as benchmarking test) Hence, there is also a need to observe seeding Realizing the importance of constraints and seeding, we are proposing a new strategy for t-way testing that is not only efficient but also cater for seeding and constraints support mechanism As part of our ongoing work [4-7], this paper describes the design and development of a t-way test data generation, called TTG In doing so, a number of experiments are discussed in order to prove the correctness of the implementation INTRODUCTION Software has become a driving strength and our society’s dependency on it, is undeniable It has been adopted in many applications including in household appliances, automotive system, sales order processing and various others As its significance grows, the software creators continually attempt to develop technologies that will make it easier, faster, and less expensive especially when it comes to constructing highquality computer programs [1] As far as dependability and reliability are concerned, the malleability of software has become an issue [2] Therefore, software testing has become an earth-shattering stage that needs to be undertaken, in order for one to identify the possible or even unexpected errors In doing so, software testing can be seen as an integral part of the software engineering lifecycle Without testing, the adoption of software in life-threatening or safety-based applications could lead to fatality Testing remains the truly effective means to assure the quality of a software system of non-trivial complexity [3], as well as one of the most intricate and least understood areas in software engineering Here, many conditions and hardware/software requirements need to be checked for acceptable conformity Although desirable, complete and exhaustive testing is impossible due to resource and timing constraints To address this issue, researchers have long relied on sampling strategies in order to partition the test data appropriately II PREVIOUS WORK Concerning constraint and seeding mechanism, most existing strategy implementations (ACS, GTWay, GA, TCG and MIPOG) have not appeared to consider the support as yet as evident by their implemented tools Strategies such as IPOG and Jenny only support constraints alone On the other hand, AETG authors claim that AETG system has the ability to support constraint and seeding, however it does not generate a minimal test set For these reasons we aim to investigate the constraints and seeding mechanism as part of our tool implementation of TTG III PROBLEM DEFINITION MODEL To illustrate the concept of t-way testing, consider an example of an online payment system to demonstrate the challenges of testing highly configurable software with two variables for every function (i.e t=2) The rationale of using this example stems from the same data input that has been used by other researchers in this area [8] T-way strategy (i.e., where t indicates the interaction strength) has been known to be effective for generating sampled test data In many situations, the sampled test data may consist of impossible or unwanted combinations Thus, there is a need to observe constraints In the same manner, there can be potentially a set of combinations that are required Figure 1: Online Payments System This research is funded by the University Technology MARA (UiTM) Excellence Fund – “Implementation of Seeding and Constraints for T-way Strategies” 978-1-4673-1310-0/12/$31.00 ©2012 IEEE 1153 2012 IEEE Symposium on Humanities, Science and Engineering Research For simplification, this online payment system options can be represented with integer numbers (see Fig 2) Here, the online payment system representation can also be translated into a table of four columns (or parameters) and rows (or values) as seen in Table I Figure 2: Online Payments System Represented Using Integer Figure 3: Merging of ABC, ABD, ACD and BCD [9] Let the input variable consist of set X = {A, B, C, D} For simplicity, assume that the starting test case has been identified and the integer numbers such as and are used to represent the data values to support the discussion Since each of four parameters has two values, the exhaustive combination would need 24 = 16 different test cases to test all possible interactions for this system TABLE I BASE DATA VALUES Input Variables Base Values A B C D 0 0 1 1 Considering t-way testing (i.e., assuming t=3), the number of combination can be reduced significantly The combination ABC, ABD, ACD and BCD are all the possible 3-way interaction (Fig 3) By merging all ABC, ABD, ACD, and BCD and merging/removing repetition, the exhaustive test case can be reduced to 14 (from a total of 16 exhaustive test cases) Going to the original problem, the following constraints can be observed • • IV PROPOSED STRATEGY, TTG In a nutshell, the TTG strategy consists of the binary combination algorithm, t-way algorithm, the seeding algorithm and the constraints algorithm The overview of TTG strategy can be represented using the flow chart as illustrated in Fig This whole algorithm exclusively utilizes a software-based work to be developed in Java language and integrated using the Netbeans IDE 6.9 software The TTG strategy considers the implementation of the higher order interactions up to t=6 (in line with the finding by [10]), with concern seeding and constraints mechanism From Fig 4, the TTG strategy involves four parts which are the main function, binary input combinations based on interaction strength, t-way test data generator and automatic interaction coverage algorithm [11] A Internet Explorer only support for Windows platform The combinations of Internet Explorer and Access will not be supported in Linux platform B Based on the aforementioned constraints, the following set of tests is deemed illegal (i.e., note that ‘x’ represents don’t care values) • {Internet Explorer, Linux,x,x} => {0,1,x,x} • {Internet Explorer,Linux,x,Access} => {0,1,x,0} Combining the two constraints, we can deduce the illegal combination as {0,1,x,x} C D Referring to Fig 3, we note that the unwanted configuration {{0,1,0,0}, {0,1,0,1}, {0,1,1,0}, {0,1,1,1}} still exists and must be removed accordingly Figure 4: The TTG Strategy Outlined In Flow Chart Part (A) describes the input parameters, degree of interaction strength, seeding and constrains parameter The 1154 2012 IEEE Symposium on Humanities, Science and Engineering Research input parameter is concerned with the input parameter value and the interaction strength The seeding algorithm involves capturing the specified test data directly into the final test suite as specified by the user and the constraints lie in the slightly more complicated algorithm and the inclusion of the test set generation algorithm inside The second part (B) concerns with the binary input combination algorithm suggested by the implementation of the interaction between parameters, aside from the fact that it generates possible combinations by considering the selection of input interaction strength Next, the third part (C) serves as an addition, where it considers the strength value in the test set generator The constraints’ algorithm iteratively finds the test case that satisfies the constraints into the final test suite The last part (D) is the automatic interaction covered algorithm showing the pair generate the coverage of all interactions and display the listing of the number of occurrences Using our earlier example of a system with parameters and values, the working of each algorithm has been discussed earlier (i.e assuming the parameter interaction strength, t=3) In our example, we will consider the selected binary combinations of ‘0111’, ‘1011’, ‘1101’ and ‘1110’ combinations Concerning the binary input combinations algorithm, the selection of ignorable values to be randomized is highly dependent on the generated binary combination list Here, the binary numbers representing the complete possible number of combination is first generated and the subsets with occurrences of three ‘1’s are selected accordingly in the binary combination list Since the parameter interaction strength is t=3, the possible combinations will be ABC, ABD, ACD and BCD Let us assume that the 3-way combination of interaction parameter is ABC The binary input combination for this interaction is {1110} The represented value of ‘D’ is not involved for this interaction but it can either be ‘0’ or ‘1’, being chosen at random The combination process of this interaction is executed in sequences, that is, until all the interactions are covered Here, the first parameter of this combination list is confirmed to be ‘A’ which contributes to this interaction and the value of this parameter will consequently be examined There are two possible values which are examined in parameter A; and Initially, the first value for parameter A will be choosen a ‘0’ and the interaction will be examined Next, the second parameter and the value of this parameter is also choosen to be ‘0’ Then, the interaction parameter for C also takes Then for the second test case, the process will follow the previous step with only difference being that parameter C will generate the second value which is This process will continue until all the combinations are completed ‘X’ is to represent the ignorable value and it generates randomly The completed cycle for the binary combination ABC totally has eight test cases (Table II) Here, this algorithm will then iterate to the next combinations (i.e ABD, ACD and BCD) The same iterations will further take place until the whole process is completed (Fig 5) Binary Input Combi nation ABD The performance of this algorithm relies on the input parameter (i.e interaction strength) and the binary input combination Then, the values involved for the first interaction parameter are examined If the effect of the test values of parameter is significant to the system (parameter with bit ‘1’ in the binary representation), the values for that parameter will be appended to the test case in sequence, only one at a time Input Variables C D A B 0 X 0 X 0 0 X X 1 X 0 X 0 X X 1 X X 0 1 X X 1 X 1 X 1 BCD TABLE II THE TEST CASE LIST FOR ABC Binary Input Combination ABC Binary Input Combination A B Binary Input Combinat ion But if the parameter is not significantly involved in the interaction (parameter with bit ‘0’ in the binary representation), random number is generated and is also appended into the test suite case Then all the complete generation will be added into the suite list For example, consider this approach (see Table II) Input Variables ACD C D Input Variables A B C D X 0 X 0 X X 1 X 0 Input Variables X 1 A B C D X 1 0 X 0 X X 1 X 0 X 1 X 1 X Figure 5: The Test case list for ABD, ACD and BDC Then, from all the combinations of the test case lists, if there are more than one test case, any repetition of test case will be deleted Finally, the Final Test Suite List will note the related combinations of the input values based on the interaction parameters Every execution from The Final Test 1155 2012 IEEE Symposium on Humanities, Science and Engineering Research Suite List will generate the different list for every execution of interaction elements since the insignificant parameter is generated randomly For the seeding algorithm, the combinations that are specified by the user are directly appended to the final test suite list The seeding parameter is stored automatically at the top of the Final Test Suite List The constraints’ algorithm is slightly more complicated than both of the algorithms given earlier Corresponding to the TTG constraints’ algorithm, the algorithm will examine the input parameter which has been determined from the binary input combinations This constraint algorithm is included in the t-way recursive loop For example, let us assume the interaction parameter is ABC, the binary input combination is {1110} and consider Table III for better understanding Now, since all pairs have been completely generated, the algorithm will check whether this pair matches the specified constraint parameter TABLE III Input Variables A ABC B C D 0 0 0 0 1 1 1 0 1 1 1 1 THE TEST SUITE LIST WITHOUT CONSTRAINT PARAMETER Binary Input Combination ABC Input Variables A B C 0 0 0 0 1 0 1 1 1 1 V DEMONSTRATION OF CORRECTNESS In order to demonstrate the correctness of the TTG strategy and to verify that all the features are described, experiments were conducted The output obtained for the above argument can be seen in Fig It is expected that the seeding’s input parameters will appear at the first level in the Final Test Suite List From Table III, note that the specified constraint parameter is assumed to be {0111} Hence, the first generated combination does not tally with the constraints’ parameter Consequently, the constraint match remains as FALSE Then check the second combination, where this generated combination also does not match the constraint parameter and again, the match remains as FALSE In the same manner, the constraints’ algorithm will continue to iterate and generate the next combinations TABLE IV Table IV shows the combinations for ABC where the unwanted pair, {0111} has been eliminated from the test suite list The constraint algorithm will then iterate to the next combinations (i.e ABD, ACD and BCD) The same iterations will be repeated until the process is complete A TTG support for seeding up to interaction strength (t) to The objective of this experiment is to demonstrate the TTG strategy support seeding for a range of interaction strength (t) up to Also, it aims to evaluate the strategy in terms of test size All the experiments were conducted in a uniform parameter (6 parameters, values) serving as the system configuration and the interaction strengths varied (t) from to with seeding of {1:1:0:1:1:0} THE MODIFIED TEST CASE LIST FOR ABC Binary Input Combination avoid unlimited loop and the algorithm will keep on looping until the limit is reached If the limit is reached, the searching loop will terminate If the constraint match is ‘FALSE’ then the combination will be added to the Final Test Suite List but if the combination matches the constraints’ parameter, it will be excluded from the Final Test Suite List D run: Input arguments: Parameter =>2,2,2,2,2,2 interaction =>6 Seeding =>1:1:0:1:1:0 -Interaction Setting => 111111 Constraint List Seeding List i = 0->1:1:0:1:1:0 Final Test Suite List i = 0->1:1:0:1:1:0 i = 1->0:0:0:0:0:0 i = 2->0:0:0:0:0:1 i = 3->0:0:0:0:1:0 i = 4->0:0:0:0:1:1 i = 5->0:0:0:1:0:0 i = 6->0:0:0:1:0:1 i = 7->0:0:0:1:1:0 i = 8->0:0:0:1:1:1 i = 9->0:0:1:0:0:0 i = 10->0:0:1:0:0:1 i = 11->0:0:1:0:1:0 i = 12->0:0:1:0:1:1 i = 13->0:0:1:1:0:0 i = 14->0:0:1:1:0:1 i = 39->1:0:0:1:1:0 i = 15->0:0:1:1:1:0 i = 40->1:0:0:1:1:1 i = 16->0:0:1:1:1:1 i = 41->1:0:1:0:0:0 i = 17->0:1:0:0:0:0 i = 42->1:0:1:0:0:1 i = 18->0:1:0:0:0:1 = 43->1:0:1:0:1:0 The iBinary Input i = 19->0:1:0:0:1:0 i = 44->1:0:1:0:1:1 Combination for i = 20->0:1:0:0:1:1 i = 45->1:0:1:1:0:0 i = 21->0:1:0:1:0:0 ABCDEF i = 46->1:0:1:1:0:1 i = 22->0:1:0:1:0:1 i = 47->1:0:1:1:1:0 i = 23->0:1:0:1:1:0 i = 48->1:0:1:1:1:1 i = 24->0:1:0:1:1:1 i = 49->1:1:0:0:0:0 i = 25->0:1:1:0:0:0 i = 50->1:1:0:0:0:1 i = 26->0:1:1:0:0:1 i = 51->1:1:0:0:1:0 i = 27->0:1:1:0:1:0 i = 52->1:1:0:0:1:1 i = 28->0:1:1:0:1:1 = 53->1:1:0:1:0:0 Seeding ivalues appear at i = 29->0:1:1:1:0:0 i = 54->1:1:0:1:0:1 the first level in the Final i = 30->0:1:1:1:0:1 i = 55->1:1:0:1:1:1 List i = 31->0:1:1:1:1:0Test i =Suite 56->1:1:1:0:0:0 i = 32->0:1:1:1:1:1 i = 57->1:1:1:0:0:1 i = 33->1:0:0:0:0:0 i = 58->1:1:1:0:1:0 i = 34->1:0:0:0:0:1 i = 59->1:1:1:0:1:1 i = 35->1:0:0:0:1:0 i = 60->1:1:1:1:0:0 i = 36->1:0:0:0:1:1 i = 61->1:1:1:1:0:1 i = 37->1:0:0:1:0:0 i = 62->1:1:1:1:1:0 i = 38->1:0:0:1:0:1 i = 63->1:1:1:1:1:1 Figure 6: Snapshot Output of TTG support seeding for interaction strength (t) to From Table III, the fourth combination generated is {0111} and it is similar to the specified constraint parameters Since the match is now found, the constraint match becomes ‘TRUE’ Next, the algorithm will check for the value of limit Here, the limit represents the maximum acceptable loops set to The output from Fig demonstrates that 64 exhaustive tests are generated for the interaction strength (t) equals to It also shows thate TTG support seeding for interaction strengths (t) For each of the interaction strength’s experiment of TTG 1156 2012 IEEE Symposium on Humanities, Science and Engineering Research support seeding, the result indicates that all seeding values are included in the Final Test Suite List Then, the result shows that the test data in the Final Test Suite List can generate the number of interaction pair coverage for each combination value Hence, the result verifies that the entire algorithm is designed correctly in order to support the t-way data generation in various degrees of the interaction strength B TTG Support for constraints up to interaction strength (t) to The objective of this experiment is to demonstrate that the TTG strategy support constraints for a range of interaction strength (t) up to Also, the aim is to evaluate this strategy in terms of test cases’ size and reduction All the experiments are conducted in a uniform parameter (6 Parameters each with values) being the system configuration, and the interaction strength varies (t) from to with constraints of {1:1:0:0:0:0,1:0:0:0:0:0} run: Input arguments: Parameter =>2,2,2,2,2,2 interaction =>6 Constraints =>1:1:0:0:0:0,1:0:0:0:0:0 -Interaction Setting => 111111 Constraint List i = 0->1:1:0:0:0:0 i = 1->1:0:0:0:0:0 Seeding List Final Test Suite List i = 0->0:0:0:0:0:0 i = 1->0:0:0:0:0:1 i = 2->0:0:0:0:1:0 i = 3->0:0:0:0:1:1 i = 4->0:0:0:1:0:0 i = 5->0:0:0:1:0:1 i = 6->0:0:0:1:1:0 i = 7->0:0:0:1:1:1 i = 8->0:0:1:0:0:0 i = 9->0:0:1:0:0:1 i = 10->0:0:1:0:1:0 i = 11->0:0:1:0:1:1 i = 12->0:0:1:1:0:0 The Binary Input i = 13->0:0:1:1:0:1 i = 14->0:0:1:1:1:0 Combination for i = 15->0:0:1:1:1:1 ABCDEF i = 16->0:1:0:0:0:0 i = 17->0:1:0:0:0:1 i = 18->0:1:0:0:1:0 i = 19->0:1:0:0:1:1 i = 20->0:1:0:1:0:0 i = 21->0:1:0:1:0:1 i = 22->0:1:0:1:1:0 i = 23->0:1:0:1:1:1 i = 24->0:1:1:0:0:0 i = 25->0:1:1:0:0:1 i = 26->0:1:1:0:1:0 Constraints values i = 27->0:1:1:0:1:1 excluded from the Final i= 28->0:1:1:1:0:0 i = 29->0:1:1:1:0:1 Test Suite List i = 30->0:1:1:1:1:0 i = 31->0:1:1:1:1:1 i = 32->1:0:0:0:0:1 i = 33->1:0:0:0:1:0 TTG supports both seeding and constraints for the interaction strength (t) up to that can be executed either in uniform or non-uniform variable parameters The objective of this experiment is to demonstrate that the TTG strategy that can support both seeding and constraints for the interaction strength (t) that varies up to It also aims to evaluate the strategy’s support for both uniform and nonuniform parameter values For this experiment, (5 Parameters each with values) is used as the system configuration on the uniform variable parameter and (3 2-valued parameters and 3-valued parameters) is used as the system configuration on the non-uniform variable parameter This experiment conducted for the interaction strength (t) that only varies to and another experiment result are shown in appendices part Here, the rationale to select this system configuration is to examine the test cases’ size on different patterns of the variable parameter 1) A uniform variable parameter (5 2-valued parameters) Here, we use 2-valued parameter with seeding of {0:1:0:0:1} and constraints of {1:1:0:0:0} It is expected that both seeding and constraints parameters given will be reflected in the final test suite list The output obtained for abovementioned configurations are illustrated in Fig Figure 7: Output of TTG support constraints for interaction strength (t) to i = 34->1:0:0:0:1:1 i = 35->1:0:0:1:0:0 i = 36->1:0:0:1:0:1 i = 37->1:0:0:1:1:0 i = 38->1:0:0:1:1:1 i = 39->1:0:1:0:0:0 i = 40->1:0:1:0:0:1 i = 41->1:0:1:0:1:0 i = 42->1:0:1:0:1:1 i = 43->1:0:1:1:0:0 i = 44->1:0:1:1:0:1 i = 45->1:0:1:1:1:0 i = 46->1:0:1:1:1:1 i = 47->1:1:0:0:0:1 These results show that the TTG can support constraints’ parameter for all the interaction strength (t) from to and automatically calculate the number of interaction coverage for each of the interaction strength For each of the experiment concerning the interaction strength of TTG support constraints, the result indicates that all constraints’ values are excluded from the Final Test Suite List Then, the result shows that the test data in Final Test Suite List can generate the number of interaction pair coverage for each combination value This can verify that the entire algorithm is designed correctly in order to support the t-way data generation in various degrees of the interaction strength i = 48->1:1:0:0:1:0 i = 49->1:1:0:0:1:1 i = 50->1:1:0:1:0:0 i = 51->1:1:0:1:0:1 i = 52->1:1:0:1:1:0 i = 53->1:1:0:1:1:1 i = 54->1:1:1:0:0:0 i = 55->1:1:1:0:0:1 i = 56->1:1:1:0:1:0 i = 57->1:1:1:0:1:1 i = 58->1:1:1:1:0:0 i = 59->1:1:1:1:0:1 i = 60->1:1:1:1:1:0 i = 61->1:1:1:1:1:1 -Figure 8: Continued Referring to Fig and 8, it is expected that the result will be a smaller sized test suite where these two constraints will be excluded from the Final Test Suite List The output from Fig and show that exhaustively 62 test sizes are generated for the interaction strength (t) to Run: Input arguments: Parameter =>2,2,2,2,2 interaction =>4 Seeding =>0:1:0:0:1 Constraints =>1:1:0:0:0 -Interaction Setting => 01111 Interaction Setting => 10111 Interaction Setting => 11011 Interaction Setting => 11101 Interaction Setting => 11110 Constraint List i = 0->1:1:0:0:0 Seeding List i = 0->0:1:0:0:1 Final Test Suite List i = 0->0:1:0:0:1 i = 1->0:0:0:0:0 i = 2->0:0:0:0:1 i = 3->0:0:0:1:0 i = 4->0:0:0:1:1 i = 5->0:0:1:0:0 i = 6->0:0:1:0:1 i = 7->1:0:1:1:0 i = 8->1:0:1:1:1 i = 9->0:1:0:0:0 i = 10->1:1:0:1:0 i = 11->1:1:0:1:1 i = 12->0:1:1:0:0 The Binary Input i = 13->0:1:1:0:1 Combination for i = 14->0:1:1:1:0 ABCDE i = 15->0:1:1:1:1 i = 16->1:0:0:0:0 i = 17->1:1:0:0:1 i = 18->1:0:1:0:0 i = 19->1:1:1:0:1 i = 20->0:0:1:1:0 i = 21->0:1:0:1:0 i = 22->0:1:0:1:1 Seeding value included i = 23->1:0:1:0:1 in the Final Test Suite i = 24->1:0:0:1:0 i = 25->1:1:1:0:0 List and constraints’ i = 26->1:1:1:1:1 value excluded from the i = 27->0:0:1:1:1 Final Test Suite List i = 28->1:0:0:1:1 i = 29->1:1:1:1:0 i = 30->1:0:0:0:1 Figure 9: Output of TTG supports both seeding and constraints using a uniform variable parameter 1157 2012 IEEE Symposium on Humanities, Science and Engineering Research 2) A non-uniform variable parameter.( 2-valued and 3-valued parameters) Here, we use a 2-valued and 3-valued parameters with seeding of {0:1:0:0:1} and constraints of {1:1:0:0:0} It is expected that both seeding and constraints parameter given will be reflected in the final test suite list The output obtains for abovementioned configurations are illustrated in Fig 10 run: Input arguments: Parameter =>2,2,2,3,3 interaction =>4 Seeding =>0:1:0:0:1 Constraints =>1:1:0:0:0 -Interaction Setting => 01111 Interaction Setting => 10111 Interaction Setting => 11011 Interaction Setting => 11101 Interaction Setting => 11110 Constraint List i = 0->1:1:0:0:0 Seeding List i = 0->0:1:0:0:1 Final Test Suite List i = 0->0:1:0:0:1 i = 1->0:0:0:0:0 i = 2->0:0:0:0:1 i = 3->1:0:0:0:2 i = 4->1:0:0:1:0 i = 5->1:0:0:1:1 i = 6->1:0:0:1:2 i = 7->0:0:0:2:0 i = 8->0:0:0:2:1 i = 9->0:0:0:2:2 i = 10->1:0:1:0:0 i = 11->1:0:1:0:1 i = 12->1:0:1:0:2 i = 13->0:0:1:1:0 i = 14->1:0:1:1:1 i = 15->1:0:1:1:2 i = 16->0:0:1:2:0 i = 17->1:0:1:2:1 i = 18->0:0:1:2:2 i = 19->0:1:0:0:0 i = 20->1:1:0:0:2 i = 21->0:1:0:1:0 i = 22->1:1:0:1:1 i = 23->1:1:0:1:2 i = 24->1:1:0:2:0 i = 25->0:1:0:2:1 i = 26->0:1:0:2:2 i = 27->1:1:1:0:0 i = 28->1:1:1:0:1 i = 29->1:1:1:0:2 The Binary Input i = 30->0:1:1:1:0 Combination for i = 31->0:1:1:1:1 i = 32->1:1:1:1:2ABCDE i = 33->1:1:1:2:0 i = 34->1:1:1:2:1 i = 35->1:1:1:2:2 i = 36->0:0:0:0:2 i = 37->0:0:0:1:0 i = 38->0:0:0:1:1 Seeding value included i = 39->0:0:0:1:2 i = 40->0:1:0:2:0 in the Final Test Suite i = 41->0:1:1:0:0 List and constraints’ i = 42->0:0:1:0:1 value excluded from the i = 43->0:0:1:0:2 i = 44->0:0:1:1:1 Final Test Suite List i = 45->0:1:1:1:2 i = 46->0:1:1:2:0 i = 47->0:0:1:2:1 i = 48->1:0:0:0:0 i = 49->1:1:0:0:1 i = 50->1:0:0:2:0 i = 51->1:0:0:2:1 i = 52->1:0:0:2:2 i = 53->1:1:1:1:0 i = 54->0:0:1:0:0 i = 55->0:0:1:1:2 i = 56->0:1:1:0:1 i = 57->0:1:1:0:2 i = 58->0:1:1:2:1 i = 59->0:1:1:2:2 i = 60->1:0:1:2:2 i = 61->1:1:1:1:1 i = 62->1:1:0:2:1 i = 63->0:1:0:1:1 i = 64->0:1:0:1:2 i = 65->1:0:0:0:1 i = 66->1:0:1:1:0 i = 67->0:1:0:0:2 i = 68->1:1:0:2:2 From the results obtained, it has been presented that the seeding mechanism at various degrees of interaction strength (equal or up to 6) is completely sustained by the TTG strategy, where the seeding parameter has successfully appeared at the first level in the Final Test Suite List, automatically suggesting that the seeding mechanism is working accurately Then, the result also presented that the constraints are also fully supported, where unwanted combinations (or constraints) parameter was excluded from the Final Test Suite List The results also obtainable the TTG strategy has been considered acceptable due to the work carried out with both uniform and non-uniform variable parameters Finally, as a scope for future work, we are now trying to benchmark TTG against some of the existing strategies REFERENCES [1] Figure 10: Output of TTG support both seeding and constraints using nonuniform variable parameter Corresponding to the actual results obtained in Fig and Fig 10, they show that TTG can address either uniform or non-uniform parameter values The results obtained show that the seeding and constraints parameters are reflected in the Final Test Suite List The seeding parameter included in the Final Test Suite List and the constraints’ parameter is excluded in the Final Test Suite List VI it is a common practice for hardware/software requirements to be checked for acceptable conformity B B Agarwal, S P Tayal & M Gupta, “Software Engineering and Testing An Introduction”, Sudbury, Jones and Bartlelt Publishers, (2010) [2] M I Younis, “Development of a Parallel T-Way Minimization Strategy for Combinatorial Testing”, Universiti Sains Malaysia, Ph D Thesis (2010) [3] V N Nair, D A James, W K Erlich, and J Zevallos, “A Statistical Assessment of Some Software Testing Strategies and Application of Experimental design Techniques Statistica Sinica”, (1998), 8(1), pp.165184 [4] M.F.J Klaib, “Development of An Automated Test Data Generation and Execution Strategy Using Combinatorial Approach”, Universiti Sains Malaysia, Ph D Thesis (2009) [5] M.I Younis, “Development of a Parallel T-Way Minimization Strategy for Combinatorila Testing”, Universiti Sains Malaysia, Ph D Thesis (2010) [6] J Md Sharif, “Implementation of Seeding and Constraints Mechanism for Pairwise Test Data Generation”, Universiti Sains Malaysia, MSc Dissertation (2010) [7] K.Z.Zamli and M.I.Younis, “Interaction Testing: From Pairwise to Variable Strength Interaction”, in Proc of the Fourth Asia International Conference on Mathematical/Analytical Modelling and Computer Simulation (AMS2010), Kota Kinabalu, pp 6-11 [8] A W Williams, “Determination of Test Configurations for Pair-wise Interaction Coverage”, Proc of the 13th International Conference on Testing of Communicating Systems 57-74 (2000) [9] K Z Zamli, N A Mat Isa, M F J Klaib, Z H Che Soh and C Z Zulkifli, “On Combinatorial Explosion Problem for Software Configuration Testing”, Proc of the Intl Conf on Robotics, Vision, Information and Signal Processing (ROVISP2007), Penang, Malaysia, 26-30 (2007) [10] D R Kuhn, D R Wallace and A M Gallo Jr., “Software fault interactions and implications for software testing Software Engineering, IEEE Transactions on Software Engineering, (2004), 30, 418-421 A Abu Bakar, “Enhancing T-way Strategy with Seeding and Constraints Support”, Universiti Sains Malaysia, MSc Dissertation (2010) CONCLUSION This research on the t-way strategy is applicable to both software and hardware systems to help achieve sound quality assurances and reliability measures Here, in many conditions, 1158 ... is to demonstrate that the TTG strategy that can support both seeding and constraints for the interaction strength (t) that varies up to It also aims to evaluate the strategy? ??s support for both... strength (t) to The objective of this experiment is to demonstrate that the TTG strategy support constraints for a range of interaction strength (t) up to Also, the aim is to evaluate this strategy. .. finds the test case that satisfies the constraints into the final test suite The last part (D) is the automatic interaction covered algorithm showing the pair generate the coverage of all interactions