Avoiding state space explosion in model checker m a thesis information technology 60 48 01

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF ENGINEERING AND TECHNOLOGY BUI PHI DIEP AVOIDING STATE-SPACE EXPLOSION IN MODEL-CHECKER MASTER THESIS OF INFORMATION TECHNOLOGY Hanoi - 2014 VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF ENGINEERING AND TECHNOLOGY BUI PHI DIEP AVOIDING STATE-SPACE EXPLOSION IN MODEL-CHECKER Major: Computer science Code: 60480101 MASTER THESIS OF INFORMATION TECHNOLOGY SUPERVISOR: Assoc Prof Nguyen Viet Ha Dr Mohamed Faouzi Atig Hanoi - 2014 Declaration of Authorship I hereby declare that this submission is my own work and to the best of my knowledge it contains no materials previously published or written by another person, or substantial proportions of material which have been accepted for the award of any other degree or diploma at University of Engineering and Technology (UET/Coltech) or any other educational institution, except where due acknowledgement is made in the thesis Any contribution made to the research by others, with whom I have worked at UET/Coltech or elsewhere, is explicitly acknowledged in the thesis I also declare that the intellectual content of this thesis is the product of my own work, except to the extent that assistance from others in the project’s design and conception or in style, presentation and linguistic expression is acknowledged Signed: Date: i ABSTRACT Model-checking is a well-known technique for the program verification problem (i.e., checking that the program satisfies a given property) However, Model-checking su↵ers the state-space explosion problem This is more visible in the the case of concurrent / parallel programs Therefore, developing new efficient techniques to address the state-space explosion problem (such as slicing) is a crucial and di fficult challenge in Model-checking In this thesis, we present a new slicing method for handling the standard state explosion problem Our slicing method consists of three steps: (1) creating an abstraction with respect to a subset of program variables, which leads to an over-approximation of the input program, (2) reconstructing a program with another subset of variables from a counterexample of abstracted program, and (3) refining the abstraction if the counterexample is a spurious one The process stops when either there are no more counterexamples remaining or there exists a counterexample after investigating all vari-ables In the former case, the program is correct; in the latter case, the program contains an error We have implemented a prototype tool and run it successfully on standard benchmarks, together with several challenging examples The experimental results show the efficiency of our method ii Acknowledgements First and foremost, I would like to express my deepest gratitude to my supervisor, Assoc.Prof Nguyen Viet Ha, for his patient guidance and continuous support throughout the years I would like to give my honest appreciation to my co-supervisor Dr Mohamed Faouzi Atig and Prof Parosh Aziz Abdulla for their great support They always appear when I need help, and respond to queries so helpfully and promptly The encourage-ment from my family, my friends in UET-VNUH and Uppsala University, and my girl friend, Diu Cap, is also very important for me When reading this thesis, if you find any mistakes, sending to me at diepbp@vnu.edu.vn is appreciated iii Contents Declaration of Authorship Acknowledgements Contents List of Figures List of Tables Introduction 1.1 1.2 1.3 Motivation Related work 1.2.1 1.2.2 Thesis structure Slicing Sequential Programs 2.1 2.2 2.3 2.4 2.5 Program Syntax Statements, Variables Program Control-Flow Graph Program Transition System Variable Slicing for Sequential Programs 2.5.1 2.5.2 Slicing Concurrent Programs 3.1 3.2 3.3 3.4 Experiment 4.1 4.2 Sequential Programs Concurrent Programs iv Contents Program Syntax Statements Program Control-Flow Graph Variable Slicing for Concurrent Programs 3.4.1 Conclusions and Future Work Bibliography List of Figures 1.1 Structure of SPIN 1.2 Our method 1.3 CEGAR framework 2.1 An example of program and its control flow graph 11 2.2 An example of program abstraction, counterexample and reconstructed program 18 2.3 Refined abstraction 20 3.1 CFG of concurrent program and their abstraction 21 3.2 A concurrent counterexample 24 3.3 Simulating the concurrent counterexample 25 4.1 A program with its initial abstraction vi 28 List of Tables 2.1 The syntax of program 2.2 Conditions on state transitions hv1, ⌦1i !P hv2, ⌦2i for each vertex type 4.1 Experimental results of verifying concurrent programs in comparison with SPIN 30 4.2 Column Information 30 4.3 Experimental results of verifying concurrent programs in comparison with tools in SV-COMP 32 vii 12 Chapter Introduction In this chapter, we describe the motivation of our work, and it is important Initially, we start with the necessaries of program verification and Model-Checking Then we state the state explosion problem, which are unavoidable by applying Model-Checking Next, we summarize our solutions and results Finally, we describe related work and thesis’s structure 1.1 Motivation Software is everywhere The appearance of software is in diverse areas namely education, healthcare system, transportation Besides, the development of multicore architecture makes software to be designed in many cores It helps software run faster, but unfortu-nately software becomes larger and more complex Software developers now need more e↵ort to not only make software but also guarantee that it works correctly Errors in software are difficult to find and fix Therefore, we need e fficient techniques, for example program verification and testing, to help us handle complex program errors Program verification is a technique that considers a program with given properties of interest It then concludes that whether the properties are satisfied in the program The considered properties of program can be valuations of variables at a program location, or no out of memory errors The result of program is either the properties are hold, which means all executions of program not violate the properties, or the properties are not hold because of a specific execution It is notable that program verification is undecidable There are no a program verification tool which can handle every type of program with every type of properties In practice, each program verification tool only focuses on a small types of program with a restricted set of properties Chapter Introduction Chapter Experiment We implemented a prototype tool using this technique using C++ The implementation currently supports verifying sequential programs and standard concurrent programs We carried out the experiment of the tool on a laptop machine with core i7 2.0 Ghz 8GB memory We tested some sequential programs with a number of program statement types Finally, we tested the concurrent programs from the benchmark by SV-COMP [18] 4.1 Sequential Programs This section contains some examples of sequential programs that our prototype tool is able to handle The programs are either correct or buggy We focus on testing whether the tool works correctly or not We then explain in details how it works and the way it avoids the state-space explosion problem Consider the program in the Figure 4.1a that has variables x0, x1, x2 The program has one assert statement at line 22 It is notable that the assert statement only considers the value of variables x2 Consequently, in order to check the program safety, we possibly only need to monitor the value of x2 In that case, we can reduce the number of monitoring states as well as memory to store states In particular, x0, x1, x2 are global variables, initialized by The program has three loops, no goto statement, so it has to pass three loops before reaching the assert state-ment at the end of program Our method starts with the variable x2 because it appears in the assert statement We abstract statements with respect to x2 to have the initial abstraction x0, x1 are 27 Chapter Experiment (a) Sequential program (b) Initial abstraction Figure 4.1: A program with its initial abstraction removed from the source The initial abstraction is shown in the Figure 4.1b The initial abstraction is simpler than the original program because the first and third loop are removed So checking the safety of the initial abstraction is simpler than checking the original one After checking the initial abstraction, SPIN generates a counterexample, which is shown in Figure 4.2a Figure 4.2b shows the reconstruction of counterexample with respect to x1, x3 The two variables are selected because there are no variables in the program having a direct relation to x2 The reconstruction consists of the first and third loop The second loop is removed because it only contains information about x2 while x2 is determined by the counterexample The assert statement is changed by replacing x2 by its value, SPIN then is used to verify the reconstruction and concludes that the reconstruction is not safe Because all variables in the program is investigated so the reconstruction corresponds to a real case of the original program The program is concluded to be unsafe and the process stops Another example is similar to the previous one with only one modification at the assert statement at line 22 The assert statement is replaced by assert (x2 == 6) The initial abstraction with respect to x2 is almost identical to Figure 4.1b with only one di↵erence in the assert statement SPIN checks the abstraction and then concludes that it is safe So the original program is safe too Chapter Experiment x2 < x2 = x2 + x2 < x2 = x2 + x2 < x2 = x2 + x2 assert(x2 == 5) Err (a) The counterexample 4.2 (b) Reconstructed program Concurrent Programs We compare our tool with SPIN and other tools in SV-COMP 2014 [23], for 14 programs in [18] Most of them are well-known programs such as Dekker [15], Lamport [16], Peterson [17], etc Table 4.1 shows our results of checking concurrent programs’ safety in comparison with SPIN • We carry out the experiment on 12 programs, including safe ones and unsafe ones These programs have about variables and threads, the number of lines is less than 100 They are quite small programs • Both our tool and SPIN return correct results for safe as well as unsafe programs If the program is unsafe, both tools can generate a counterexample That means our tool can find difficult bugs in concurrent programs and prove programs working correctly • The experimental results of SPIN show that it is really good at small programs with a few variables and threads In most programs, SPIN only takes 0.1 second to compile and return the result Chapter Experiment Table 4.1: Experimental results of verifying concurrent programs in comparison with Program Dekker safe Lamport safe Peterson safe Szymanski safe Read-Write safe Time Var Mutex safe Lazy safe Stateful safe Stateful unsafe Twostage unsafe Reorder unsafe Sync unsafe • Our tool generally takes more compilation and verification time than SPIN in these programs The reason is that our method ideally splits a program to many smaller programs and verifies them So our tool has to compile many programs while SPIN only does once For small programs, splitting a program to many smaller ones is not really efficient The Table 4.3 shows results of our tool in comparison with other tools, namely CBMC [19], Lazy-CSeq [20], MU-CSeq [21] and LLBMC [22], in Competition on Software Ver-ification, SV-COMP 2014 [23] The tools run in the benchmark we described above • CBMC 4.5, Lazy-CSeq, MU-CSeq and our tool return correct results, including safe programs and unsafe programs, while LLBMC cannot detect the programs safety LLBMC returns unknown for all programs, that means it cannot check these programs Chapter Experiment • In most tests, our tool is faster than other tools Particularly, our tool is the fastest one when experiencing 7/12 tests CBMC is the fastest one in 5/12 tests However, our tool does not have good results at two tests, Read-Write safe and Time Var Mutex safe, because these program have a huge number of counterexamples, that means we have to handle many small programs Reor Sync Table 4.3: Experimental results of verifying concurrent programs in comparison with tools in SV-COMP Program Dekker safe Lamport safe Peterson safe Szymanski safe Read-Write safe Time Var Mutex safe Lazy safe Stateful safe Stateful unsafe Twostage unsafe Chapter Experiment 32 Chapter Conclusions and Future Work We have presented a new slicing method for automatic checking program safety Our method focuses on handling the state-space explosion problem, which is unavoidable by applying model checking We have implemented our method in a prototype tool and applied it to several challenging examples In particular, we have checked famous programs with difficult bugs The experiment results show the e fficiency of our method Our work opens several directions: • Applying a new predicate abstraction to reduce the number of counterexample of each abstraction of program Similarly, by using a new predicate abstraction, we would have a better refinement: removing a counterexample from an abstraction means removing a class of counterexamples which not correspond to the original program • Developing a new model checker which can be integrated with our method So we would not re-compile the abstraction when we make slight changes in the abstrac-tion 33 Bibliography [1] Michael Grottke and Kishor S Trivedi Fighting bugs: Remove, retry, replicate, and rejuvenate Computer, 40(2):107–109, February 2007 ISSN 0018-9162 doi: 10.1109/MC.2007.55 URL http://dx.doi.org/10.1109/MC.2007.55 [2] Edmund Clarke, Orna Grumberg, Somesh Jha, Yuan Lu, and Helmut Veith Counterexample-guided abstraction refinement for symbolic model checking J ACM, 50(5):752–794, September 2003 ISSN 0004-5411 doi: 10.1145/876638 876643 URL http://doi.acm.org/10.1145/876638.876643 [3] Edmund M Clarke 25 years of model checking chapter The Birth of Model Checking, pages 1–26 Springer-Verlag, Berlin, Heidelberg, 2008 ISBN 978-3540-69849-4 doi: 10.1007/978-3-540-69850-0 URL http://dx.doi.org/10.1007/ 978-3-540-69850-0_1 [4] Gerard J Holzmann The model checker spin IEEE Trans Softw Eng., 23(5): 279–295, May 1997 ISSN 0098-5589 doi: 10.1109/32.588521 URL http://dx doi.org/10.1109/32.588521 [5] Gerard J Holzmann Design and Validation of Computer Protocols PrenticeHall, Inc., Upper Saddle River, NJ, USA, 1991 ISBN 0-13-539925-4 [6] Amir Pnueli The temporal logic of programs In Proceedings of the 18th Annual Symposium on Foundations of Computer Science, SFCS ’77, pages 46–57, Washing-ton, DC, USA, 1977 IEEE Computer Society doi: 10.1109/SFCS.1977.32 URL http://dx.doi.org/10.1109/SFCS.1977.32 [7] Mark Weiser Program slicing In Proceedings of the 5th International Conference on Software Engineering, ICSE ’81, pages 439–449, Piscataway, NJ, USA, 1981 IEEE Press ISBN 0-89791-146-6 URL http://dl.acm.org/citation.cfm?id= 800078.802557 [8] Hiralal Agrawal and Joseph R Horgan Dynamic program slicing SIGPLAN Not., 25(6):246–256, June 1990 ISSN 0362-1340 doi: 10.1145/93548.93576 URL http://doi.acm.org/10.1145/93548.93576 34 Chapter Experiment [9] Gerardo Canfora, Aniello Cimitile, and Andrea De Lucia Conditioned program slicing Information and Software Technology, 40(11–12):595 – 607, 1998 ISSN 0950-5849 doi: http://dx.doi.org/10.1016/S0950-5849(98)00086-X URL http: //www.sciencedirect.com/science/article/pii/S095058499800086X [10] Mary Jean Harrold Rajiv Gupta, Mary Jean and Mary Lou So↵a An approach to regression testing using slicing In In Proceedings of the Conference on Software Maintenance, pages 299–308 IEEE Computer Society Press, 1992 [11] Keith Brian Gallagher and James R Lyle Using program slicing in software main-tenance IEEE Trans Softw Eng., 17(8):751–761, August 1991 ISSN 0098-5589 doi: 10.1109/32.83912 URL http://dx.doi.org/10.1109/32.83912 [12] Susanne Graf and Hassen Saidi Construction of abstract state graphs with pvs In Proceedings of the 9th International Conference on Computer Aided Verification, CAV ’97, pages 72–83, London, UK, UK, 1997 Springer-Verlag ISBN 3-540-63166-6 URL http://dl.acm.org/citation.cfm?id=647766.733618 [13] Michael Colon and Tomas E Uribe Generating finite-state abstractions of reactive systems using decision procedures In Proceedings of the 10th International Confer-ence on Computer Aided Verification, CAV ’98, pages 293– 304, London, UK, UK, 1998 Springer-Verlag ISBN 3-540-64608-6 URL http://dl.acm.org/citation cfm?id=647767.733631 [14] Edmund M Clarke, Orna Grumberg, and David E Long Model checking and abstraction ACM Trans Program Lang Syst., 16(5):1512–1542, September 1994 ISSN 0164-0925 doi: 10.1145/186025.186051 URL http://doi.acm.org/10 1145/186025.186051 [15] Edsger W Dijkstra The origin of concurrent programming chapter Cooperating Sequential Processes, pages 65–138 Springer-Verlag New York, Inc., New York, NY, USA, 2002 ISBN 0-387-95401-5 URL http://dl.acm.org/citation.cfm? id=762971.762974 [16] Leslie Lamport A fast mutual exclusion algorithm ACM Trans Comput Syst., 5(1):1–11, January 1987 ISSN 0734-2071 doi: 10.1145/7351.7352 URL http: //doi.acm.org/10.1145/7351.7352 [17] Gary L Peterson Myths about the mutual exclusion problem Inf Process Lett., 12(3):115–116, 1981 doi: 10.1016/0020-0190(81)90106-X URL http://dx.doi org/10.1016/0020-0190(81)90106-X [18] Dirk Beyer Second competition on software verification (summary of sv-comp 2013), 2013 Chapter Experiment [19] Daniel Kroening and Michael Tautschnig Cbmc – c bounded model checker In Erika Abrah´am and Klaus Havelund, editors, Tools and Algorithms for the Construction and Analysis of Systems, volume 8413 of Lecture Notes in Computer Science, pages 389–391 Springer Berlin Heidelberg, 2014 ISBN 978-3642-54861-1 doi: 10.1007/978-3-642-54862-8 26 URL http://dx.doi.org/10.1007/ 978-3-642-54862-8_26 [20] Omar Inverso, Ermenegildo Tomasco, Bernd Fischer, Salvatore La Torre, and Gennaro Parlato Lazy-cseq: A lazy sequentialization tool for c In Erika Abrah´am and Klaus Havelund, editors, Tools and Algorithms for the Construction and Anal-ysis of Systems, volume 8413 of Lecture Notes in Computer Science, pages 398– 401 Springer Berlin Heidelberg, 2014 ISBN 978-3-642-54861-1 doi: 10.1007/ 978-3-642-54862-8 29 URL http://dx.doi.org/10.1007/978-3-642-54862-8_ 29 [21] Ermenegildo Tomasco, Omar Inverso, Bernd Fischer, Salvatore La Torre, and Gennaro Parlato Mu-cseq: Sequentialization of c programs by shared memory unwindings In Erika Abrah´am and Klaus Havelund, editors, Tools and Algorithms for the Construction and Analysis of Systems, volume 8413 of Lecture Notes in Computer Science, pages 402–404 Springer Berlin Heidelberg, 2014 ISBN 978-3-642-54861-1 doi: 10.1007/978-3-642-54862-8 30 URL http://dx.doi.org/10 1007/978-3-642-54862-8_30 [22] Carsten Sinz, Florian Merz, and Stephan Falke Llbmc: A bounded model checker for llvm’s intermediate representation In Cormac Flanagan and Barbara Kăonig, editors, Tools and Algorithms for the Construction and Analysis of Systems, vol-ume 7214 of Lecture Notes in Computer Science, pages 542–544 Springer Berlin Heidelberg, 2012 ISBN 978-3-642-28755-8 doi: 10.1007/978-3642-28756-5 44 URL http://dx.doi.org/10.1007/978-3-642-28756-5_44 [23] Dirk Beyer Status report on software verification In Erika Abrah´am and Klaus Havelund, editors, Tools and Algorithms for the Construction and Analysis of Systems, volume 8413 of Lecture Notes in Computer Science, pages 373–388 Springer Berlin Heidelberg, 2014 ISBN 978-3-642-54861-1 doi: 10.1007/978-3-642-54862-8 25 URL http://dx.doi.org/10.1007/978-3-642-54862-8_25 ...VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF ENGINEERING AND TECHNOLOGY BUI PHI DIEP AVOIDING STATE- SPACE EXPLOSION IN MODEL- CHECKER Major: Computer science Code: 604 8010 1 MASTER THESIS OF INFORMATION. .. predicates So each abstract state in the abstracted program models a number of state in the original program There exists a transition from an abstract state if at least one corresponding concrete... programs in comparison with tools in SV-COMP Program Dekker safe Lamport safe Peterson safe Szymanski safe Read-Write safe Time Var Mutex safe Lazy safe Stateful safe Stateful unsafe Twostage unsafe