An eficient method for automated control flow testing of programs : M.A Thesis Information Technology : 60 48 01

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF ENGINEERING AND TECHNOLOGY Nguyen Quang Trung AN EFFICIENT METHOD FOR AUTOMATED CONTROL FLOW TESTING OF PROGRAMS MASTER THESIS OF INFORMATION TECHNOLOGY hA nOI - 2014 VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF ENGINEERING AND TECHNOLOGY Nguyen Quang Trung AN EFFICIENT METHOD FOR AUTOMATED CONTROL FLOW TESTING OF PROGRAMS Major: Computer science Code: 60 48 01 MASTER THESIS OF INFORMATION TECHNOLOGY Supervisor: Dr Pham Ngoc Hung HA NOI - 2014 AUTHORSHIP “I hereby declare that the work contained in this thesis is of my own and has not been previously submitted for a degree or diploma at this or any other higher education institution To the best of my knowledge and belief, the thesis contains no materials previously published or written by another person except where due reference or acknowledgement is made.” Signature:……………………………………………… i SUPERVISOR’S APPROVAL “I hereby approve that the thesis in its current form is ready for committee examination as a requirement for the Bachelor of Computer Science degree at the University of Engineering and Technology.” Signature:……………………………………………… ii ACKNOWLEDGEMENT First of all, I would like to express my sincere gratitude to my supervisor Dr Pham Ngoc Hung, University of Engineering and Technology, Viet Nam National University (VNU), Ha Noi for his enthusiastic guidance, warm encouragement and helpful research experiences I am grateful to thank all the teachers in University of Engineering and Technology, VNU who provide invaluable knowledge and life skills for me during the four academic years I would like to also thank my friends in K18CS class who helped me during the two academic years Last, but not least, my family is really the biggest motivation for me My parents always encourage me when I have stress and difficulty I would like to send them great love and gratefulness Ha Noi, May …, 2014 Nguyen Quang Trung iii List of Figures Figure 2.1 Black-box model processing Figure 2.2 An example about white-box testing Figure 2.3 Control flow graph generated from the program Figure 2.4 Source code, control flow graph and ddgraph of tree method 15 Figure 2.5 Source code, control flow graph and ddgraph of tree method 16 Figure 3.1 Process of control flow testing .20 Figure 3.2 Siplifying process of the ddgraph 23 Figure 3.3 Finding process 25 Figure 4.1 Code and ddgraph of calcutateBill method 29 Figure 4.2 Ddgraph after simplifying 30 iv List of Tables Table 4.1 Black-box input 29 Table 4.2 White-box input 31 Table 4.3 Symbolic execution of path 33 Table 4.4 Symbolic execution of path 35 v Table of Contents List of Figures iv List of Tables v Abstract viii Chapter INTRODUCTION .1 Chapter AUTOMATED TESTING APPROACHES 2.1 Software quality 2.2 Role of testing 2.3 Testing Techniques 2.3.1 Black-Box Testing 2.3.2 White-box Testing 2.3.3 Grey Box Testing .9 2.4 Types of level testing 10 2.4.1 Component Testing 10 2.4.2 Integration Testing 11 2.4.3 System Testing 12 2.4.4 Acceptance Testing 13 2.5 Control Flow Testing .13 2.5.1 Definition 14 2.5.2 Supporting definition and theorem 16 Chapter AN EFFICIENT METHOD FOR AUTOMATED CONTROL FLOW TESTING OF PROGRAMS 18 3.1 Test data 18 3.2 Generating simplified Ddgraph .20 3.3 Generating Test Input Data 23 3.4 Time complexity 25 vi Chapter TOOL AND CASE STUDY .28 Chapter CONCLUSION 36 References 38 vii Abstract The main goals of software testing are to reveal bugs and to ensure that the system being developed complies with the customer’s requirements To make testing effective, it is recommended that test planning/development begin at the onset of the project Software testing techniques can be divided into approaches: black-box and white-box techniques Black-box testing is mainly a validation technique that checks to see if the product meets the customer’s requirements On the other hand, white-box testing is a verification technique which uses the source code to guide the selection of test data Currently, software companies focus only on the black-box testing techniques in order to validate because White-box testing very expensive, slow and it requires the tester have knowledge of code In addition, implementing black-box and white-box testing separately makes white-box testing becomes more inefficient This thesis presents a method for automated control flow testing of unit programs to solve the problems The key idea of this method is to combine the black-box and white-box techniques in order to minimize the complexity of white-box testing Instead of performing black-box and white-box separately, the proposed method uses the test inputs that are generated by black-box to reduce searching space of white-box testing The method then continually eliminates arcs in remaining space to find non-duplicated test paths Therefore, the proposed method is able to operate white-box testing with less effort than the current method Key words: Automated Testing, Control flow testing, black-box testing, white-box testing, Java testing, Eclipse plugin viii P2 = (e1; e3; e6) Finally, after reducing arc e4 and with ddgraph having only one arc e 1;2;5 , the finding process is done The process returns set of paths S = {(e1; e3; e7); (e1; e3; e6); (e1; e2; e4); (e1; e2; e5)} covering all branches of the ddgraph Figure 3.3 Finding process 3.4 Time complexity In this section, we estimate the time complexity of the proposed method It can be calculated based on the time complexity of two major Algorithms and The time complexity of the method depends on many factors, such as the number of decision nodes, the shape of graph, and the coverage of black-box test cases In the worst cases, if simplifying process is not operated, it may be approximately O(n 4) In almost cases, when combining black-box, the time complexity of the proposed method is approximately O(n 2log(n)) The following explains how the proposed method can reach time complexity O(n 2log(n)) At first glance, the time complexity of the proposed method is the maximum of the simplifying process’s time complexity and the selecting process’s one So, the time complexity of generating simplified ddgraph and the one of generating simplified ddgraph are considered, respectively 25 First, the generating simplified ddgraph is presented in Algorithm In the while loop of Algorithm 1, we can easily see that the codes from line to line which reduce redundant arcs have time complexity O(n) And, O(n) is the time complexity of codes eliminating not existed arcs from line to 13 ADJUST1 procedure presented in Algorithm has two loops Thus, ADJUST1’s time complexity is approximately O(n 2) The SIMPLIFY procedure’s time complexity depends on how many arcs in the visited list In practice, when black-box’s path covers all branches of the ddgraph, the SIMPLIFY procedure is not operated But in order to estimate time complexity, we can assume that the SIMPLIFY procedure is performed when all branches are visited So, in the worst case, in Fig 2.1(c), when ddgraph has shape like complete binary tree, in each round we can reduce from the height of tree when half of arcs which are connected to ending node are removed Height H of tree is calculated as follows: H = log (|N |−1)+1 So we have time complexity of SIMPLIFY procedure is approximately O(n 2log (n)) In the FIND BASIC procedure presented in Algorithm 3, the time complexity depends on how many arcs can be reduced from the path 26 found by DFS With the best case, the ddgraph which has series of condition nodes, the FIND BASIC procedure is performed just one time On the contrary, in the worst case, with ddgraph such as figure 1(c), we can reduce one arc from a found path and the loop will be repeated |N |−1 times Furthermore, depth-first searching algorithm is executed repeatedly In each Depth-first searching loop, we use ADJUST2 procedure so that time complexity can be calculated as O(|A|)*O(|N|*|A|), approximate O(n3) Hence, when being performed just one time in the best case, it has O(n 3) time complexity And in the worst case, it has O(n 4) time complexity 27 Chapter TOOL AND CASE STUDY This chapter presents case study for the proposed method in order to show the effectiveness of the proposed method Although the FIND BASIC procedure needs a lot of time but after being simplified, the size of ddgraph is very small We can examine the example of calculateBill method in Java This method implements calculating bill task of restaurant The inputs of this method are usage, member’s score and date The Figure 4.1 shows the Java code and the ddgraph of calculateBill method The first step of the proposed method simplifies the original ddgraph By dint of using the symbolic execution we can find the arcs covered by black-box testing The Table 4.1 shows that 10 sets of black-box test inputs and their covered paths No Usage 0 100 400 600 Member 50 Score 150 50 150 50 Date 25 25 Path e1, e3, e19 e1, e3, e18 e1, e2, e5, e6 e1, e2, e5, e7, e8 e1, e2, e4, e11, e13, e17 28 10 600 600 600 600 600 50 25 e1, e2, e4, e11, e12, e15 150 e1, e2, e4, e11, e13, e16 150 25 e1, e2, e4, e11, e12, e15 300 e1, e2, e4, e11, e13, e16 300 25 e1, e2, e4, e11, e12, e14 Table 4.1 Black-box input Figure 4.1 Code and ddgraph of calcutateBill method And the Figure 4.2 shows the ddgraph after reduced redundant arcs which are covered by black-box test cases 29 Figure 4.2 Ddgraph after simplifying As a result, the ddgraph of this method was declined from graph having 11 nodes, 19 arcs to graph having nodes and arcs Thus, whitebox has to find two paths S={(e1; e2; e4; e10); (e1; e2; e5; e7; e9)} for covering all branches of the sub graph In summary, the final result is that the complexity time of whole process is approximately O(n 2log (n)) The symbolic execution is called again to find path’s respective constraint The Table 4.2 shows the result of the while-box’s test inputs corresponding with the test paths and constraints of them No Path Constraints Test input e1, e2, e4, e10 Usage > Usage = 500 Usage ≥ 500 ((40) + 50 + Usage) ≤ 600 e1, e2, e5, e7, e9 Usage > Usage = 201 Usage < 500 Usage > 200 ((40) + 50 + Usage + (Usage -50)*0,1) < 400 30 Table 4.2 White-box input Table 4.3 and 4.4 shows the symbolic execution with each expression and their state Expression State Usage=Usage Usage=Usage memberScore=memberSco re Usage = Usage date=date memberScore = memberScore Usage = Usage memberScore = memberScore date = date Bill=0 Usage = Usage memberScore = memberScore date = date Bill = Usage>0 Usage = Usage memberScore = memberScore date = date Bill = 40 Bill=40 Usage = Usage 31 memberScore = memberScore date = date Bill = 40 Bill=40 Usage = Usage memberScore = memberScore date = date Bill = 40 Usage>=500 Usage = Usage memberScore = memberScore date = date Bill = 40 discount=0 Usage = Usage memberScore = memberScore date = date Bill = 40 discount = Bill=Bill+50+Usage Usage = Usage memberScore = memberScore date = date Bill = (40)+50+Usage 32 discount = Bill=Bill-Bill*discount/100 Usage = Usage memberScore = memberScore date = date Bill=(40)+50+Usage)((40)+50+Usage)*discount/100 discount = return=Bill Usage = Usage memberScore = memberScore date = date Bill=((40)+50+Usage)((40)+50+Usage)*discount/100 discount = Table 4.3 Symbolic execution of path Expression State Usage=Usage Usage=Usage memberScore=memberScore Usage = Usage memberScore = memberScore 33 date=date Usage = Usage memberScore = memberScore date = date Bill=0 Usage = Usage memberScore = memberScore date = date Bill = Usage200 Usage = Usage memberScore = memberScore date = date Bill = 40 Bill=Bill+50+Usage+(Usage50)*0.1 Usage = Usage memberScore = memberScore date = date Bill = (40)+50+Usage+(Usage-50)*0.1 34 return=Bill Usage = Usage memberScore = memberScore date = date Bill = (40)+50+Usage+(Usage-50)*0.1 Table 4.4 Symbolic execution of path 35 Chapter CONCLUSION This thesis has been completed objective and satisfied the requirements of subjective: a method for automated generate white-box testing input The key idea of this method is to generate all test cases such that they cover all path of Java program We have applied the automated tool to generate input data for simple Java method In this thesis, we built a test case for our tool such as calculator bill function and obtained very good results The obtained experimental results are clear to show that the implemented tool can detect several paths that are not covered by blackbox test cases with their inputs Thank to input data generated automatically, testers who have not knowledge about code, now, can perform white-box testing more quickly and effectively The company will save a lot of time and cost for white-box testing by using the available black-box input data As show above, with all black-box’s test paths eliminated, the control flow testing significantly reduces complexity The time complexity of this method declines from O(n4) when performing white-box separately to O(n2log(n)) when combining with black-box In addition, by continually removing the redundant arcs and nodes after finding complete paths, the proposed method can generate set of test inputs, which followed by yielding non duplicated paths As a result, by take full advantage of black36 box result, this method can select a small set of test inputs that cover all branches of graph with little effort Does this method not only propose an approach get most of the black-box, it is also completely operated to reuse test cases in the context of program evolution Albeit, the proposed method has not supported for program unit with loop yet but in future, for solving this problem, we will focus on detecting and separating loop from graph to present it under no loop graph form Now, this method is being implemented as a plugin of Eclipse for control flow testing Java unit In the next time, we will complete the plugin for evaluating performance of this method in practice 37 References T.McCabe, J Thomas, -‖Structural Testing: A Software Testing Methodology Using the Cyclomatic Complexity Metric‖, NIST Special Publication 500-99, Washington D.C., 1982 A Bertolino, M Marre, - ‖Automatic Generation of Path Covers Based on the Control flow analysis of computer Programs‖ IEEE Transaction on Software Engineering, vol.20, no.12, pp.885-899, 1994 Z Guangmei, C Rui, L Xiaowei, H Congying, - ‖The Automatic Generation of Basis Set of Path for Path Testing‖, Proceedings of the 14th Asian Test Symposium (ATS ’05), 2005 4.Martina Marre, Antonia Bertolino, - ‖Using Spanning Sets for Coverage Testing‖, IEEE Transaction on Software Engineering, vol.29, no.11, pp.974-984, 1993 R Gupta and M.L Soffa, ‖Employing Static Information in the Generation ofTest Cases,‖ Software Testing, Verification and Reliability, vol 3, no 1, pp.29-48,1993 AhmedS Ghiduk, O Said and Sultan Aljahdali, ”Basis Test Paths Generation Using Genetic Algorithm,” International Conference on Computing The Information Technology (ICCIT), pp.303-308, 2012 Jame C.King, -”Symbolic Execution and Program Testing” Communications of The ACM, vol.19, no.7, pp.385-394, 1976 D A Garvin What Does “Product Quality” Really Mean?, Sloan Management Review, Fall 1984, pp 25 – 43 L Osterweil et al Strategic Directions in Software Quality ACM Computing Surveys, December 1996, pp 738 – 750 38 10 M A Friedman and J.M Voas Software Assessment: Reliability, Safety, Testability Wiley, NewYork, 1995 11 Michael D Ernst Static and Dynamic Analysis: Synergy and Duality Paper presented at ICSE Workshop on Dynamic Analysis, Portland, OR, May 2003, pp 24 – 27 12 Ron, Patton Software Testing November 7, 2000 13 Standard for Software Component Testing, Working Draft 3.4, 27 April 2001 14 Software Testing Fundamentals 2011-11-04 15 ISTQB Foundation Level (CTFL) Syllabus 2011 16 Feathers, M Working Effectively with Legacy Code, Prentice Hall, 2004 17 Newkirk, JW and Vorontsov, AA Test-Driven Development in Microsoft.NET, Microsoft Press, 2004 18 Ricardo Simões APAC: An exact algorithm for retrieving cycles and paths in all kinds of graphs 19 B.Kitchenham and S L Pfleeger Software Quality: The Elusive Target.IEEE Software, January 1996, pp 12 – 21 39