DSpace at VNU: Assume-guarantee tools for component-based software verification

2010 Second International Conference on Knowledge and Systems Engineering Assume-Guarantee Tools for Component-Based Software Verification Pham Ngoc Hung Viet-Ha Nguyen College of Technology, Vietnam National University, Hanoi {hungpn, hanv}@vnu.edu.vn Abstract This paper presents a minimized assumption generation method and its associated tools for L*-based assumeguarantee verification of component-based software by model checking The method is not only fitted to componentbased software but also has a potential to solve the state space explosion problem in model checking In the proposed method, a verification target is decomposed into components so that we can model check each of them separately The key idea of this method is finding the minimal assumptions in the search spaces of the candidate assumptions The minimal assumptions generated by the proposed method can be used to recheck the whole system at much lower computational cost Our experience so far indicates that the implemented tools are potential for verifying practical component-based software Introduction Although model checking has been recognized as a promising approach for improving software reliability, a major problem is the state space explosion for applying large-scale systems [2] The assume-guarantee approach proposed in [9, 13, 14] is a potential solution to deal with the problem It is a “divide and conquer” mechanism for breaking up the verification task of a component-based system (CBS) into smaller tasks that involve the verification of its individual components The L*-based assumption generation method proposed in [3] is a promising, incremental and fully automatic approach for modular verification of component-based software This work proposes an iterative approach based on a learning algorithm called L* [1, 15] for learning an assumption that is the environment needed for a component to satisfy a property and for the rest of the CBS to be satisfied However, the assumptions generated by this method are not minimal [7] Thus, the number of states of the as978-0-7695-4213-3/10 $25.00 © 2010 IEEE DOI 10.1109/KSE.2010.18 Toshiaki Aoki Takuya Katayama School of Information Science, Japan Advanced Institute of Science and Technology {toshiaki, katayama}@jaist.ac.jp sumptions should be minimized because the computational cost of model checking is influenced by that number Moreover, when a component is evolved in the context of the software evolution, the whole evolved CBS of many existing components and the evolved component is required to be rechecked [5, 6, 8] In this case, we also can reduce the cost of rechecking the evolved CBS by reusing the smaller assumption The facts imply that the sizes of the generated assumptions are of primary importance Furthermore, although the assume-guarantee verification has been received a widespread attention from the software engineering community, there is not any available tool which supports this approach Thus, an available tool for the approach should be implemented We have proposed a method for generating minimal assumptions for the assume-guarantee verification of CBS in [7] The key idea of this method is finding the minimal assumptions that satisfies the assume-guarantee rules in the search spaces of the candidate assumptions by using the breadth-first search algorithm The generated assumptions are seen as the environments needed for components to satisfy a property and for the rest of the CBS to be satisfied In order to evaluate the effectiveness of the proposed method, we have implemented two tools for the assumption generation method proposed in [3] and the proposed method respectively Some typical CBS are applied to compare our method with that proposed in [3] Our obtained experimental results shows that the our method can generate the minimal assumptions which have the minimal sizes and a smaller number of transitions than the assumptions generated by the method proposed in [3] These minimal assumptions generated by the proposed method can be used to recheck the whole CBS by checking the assume-guarantee rules at much lower computational costs The experiment also implies that our method is potential for verifying largescale CBS The rest of this paper is organized as follows We first describes the current method for assumption generation by using the L* learning algorithm in Sect Section is about 172 our minimized assumption generation method to find the minimal assumptions for component-based software verification Section discusses the tool support for our method and experimental results obtained from the application of our method to some typical case studies Finally, we conclude the paper in Sect L*-Based Assumption Generation Method 2.1 The L* Learning Algorithm The L* learning algorithms was developed by Angluin [1] and later was improved by Rivest and Schapire [15] L* learns an unknown regular language and produces a deterministic finite state automata (DFA) that accepts it The main idea of the L* learning algorithms is based on the “Myhill-Nerode Theorem” [11] in the theory of formal languages It said that for every regular set U⊆ Σ∗ where Σ is the alphabet, there exists a unique minimal deterministic automata whose states are isomorphic to the set of equivalence classes of the following relation: w ≈w iff ∀u ∈ Σ∗ : wu ∈ U ⇐⇒ w u ∈ U Therefore, the main idea of L* is to learn the equivalence classes, i.e., two prefix are not in the same class if and only if there is a distinguishing suffix u [7] Let U be an unknown regular language over some alphabet Σ L* will produce a DFA M such that M is a minimal deterministic automata corresponding to U and L(M ) = U In order to learn U , L* needs to interact with a Minimally Adequate Teacher, called Teacher The Teacher must be able to correctly answer two types of questions from L* The first type is a membership query, consisting of a string σ ∈ Σ∗ ; the answer is true if σ ∈ U , and f alse otherwise The second type of these questions is a conjecture, i.e., a candidate DFA M whose language the algorithm believes to be identical to U The answer is true if L(M ) = U Otherwise the Teacher returns a counterexample, which is a string σ in the symmetric difference of L(M ) and U At a higher level, L* maintains a table T that records whether string s in Σ∗ belong to U It does this by making membership queries to the Teacher to update the table At various stages L* decides to make a conjecture It uses the table T to build a candidate DFA Mi and asks the Teacher whether the conjecture is correct If the Teacher replies true, the algorithm terminates Otherwise, L* uses the counterexample returned by the Teacher to maintain the table with string s that witness differences between L(Mi ) and U 2.2 L*-Based Method Assumption Generation The assume-guarantee paradigm is a powerful “divideand-conquer” mechanism for decomposing a verification process of a CBS into subtasks about the individual components Consider a simple case where a system is made up of two components including a framework M1 and an extension M2 The goal of this approach is to verify whether this system satisfies a property p without composing M1 with M2 For this purpose, if there exists an assumption A(p) such that A(p) is strong enough for M1 to satisfy p but weak enough to be discharged by M2 (i.e., A(p) M1 p and true M2 A(p) which are called assume-guarantee rules, both hold), the compositional system M1 M2 satisfies p Unfortunately, it is often difficult to find such assumption An assumption generation method proposed in [3] applies the assume-guarantee rules in an iterative approach illustrated in Fig At each iteration i, a candidate assumption Ai is produced based on some knowledge about the system and the results of the previous iteration The two steps of the assume-guarantee rules are then applied Step checks whether M1 satisfies p in an environment that guarantees Ai by computing formula Ai M1 p If the result is f alse, it means that this candidate assumption is too weak The candidate assumption Ai therefore must be strengthened with the help of the counterexample cex produced by this step Otherwise, the result is true, it means that Ai is strong enough for the property to be satisfied The step is then applied to check that if component M2 satisfies Ai by computing formula true M2 Ai If this step returns true, the property p holds in the compositional system M1 M2 and the algorithm terminates Otherwise, this step returns f alse; further analysis is required to identify whether p is indeed violated in M1 M2 or the candidate Ai is too strong to be satisfied by M2 Such analysis is based on the counterexample cex returned by this step The L* algorithm must check that the counterexample cex belong to the unknown language U = L(AW ) If it does not, the property p does not hold in the system M1 M2 Otherwise, Ai is too strong The candidate assumption Ai must be weakened (i.e., behaviors must be added with the help of cex) in iteration i + A new candidate assumption may of course be too weak, and therefore the entire process must be repeated Minimized Method Assumption Generation We have proven that the assumptions generated by the L*-based assumption generation method proposed in [3] are not minimal (i.e., the numbers of states of the generated as- 173 Figure A framework for L*-based assumption generation sumption are not smallest) Thus, we have proposed the method for generating minimal assumptions for the assumeguarantee verification of component-based software presented in [7] In this method, finding a minimal assumption is considered as a search problem in a search space of all candidate assumptions The proposed method is a combination of the L* learning algorithm and the breadth-first search algorithm We use the breadth-first search strategy because this strategy ensures that the generated assumption is minimal (see Theorem in [7]) In order to learn a candidate assumption, L* builds an observation table (S, E, T ) defined as follows: Definition (Observation table) (S, E, T ) is an observation table built by L*, where: • S ⊆ Σ∗ is a set of prefixes It presents equivalence classes or states • E ⊆ Σ∗ is a set of suffixes It presents the distinguishing • T : (S ∪ S.Σ).E → {true, f alse} where, the operator “.” means that given two sets of event sequences P and Q, P Q = {pq | p ∈ P, q ∈ Q}, where pq presents the concatenation of the event sequences p and q With a string s in Σ∗ , T (s) = true means s ∈ U , otherwise s ∈ U Remark An observation table (S, E, T ) is closed if ∀s ∈ S, ∀a ∈ Σ, ∃s ∈ S, ∀e ∈ E: T (sae) = T (s e) In this case, s presents the next state from s after seeing a, sa is undistinguishable form s by any of suffixes Intuitively, the observation table (S, E, T ) is closed means that every row sa of S.Σ has a matching row s in S use a queue data structure which contains the generated observation tables with the first-in first-out order These observation tables are used for generating the candidate assumptions Initially, the algorithm sets the queue q to the empty queue (line 1) We then put the initial observation table OT0 = (S0 , E0 , T0 ) into the queue q as the root of the search space of observation tables, where S0 = E0 = {λ} (λ represents the empty string) (line 2) Subsequently, the algorithm gets a table OTi from the top of the queue q (line 4) If OTi contains the “don’t know” value “?” (line 5), we obtain all instances of OTi by replacing all “?” entries in OTi with both true and f alse (line 6) The obtained instances then are put into the queue q (line 7) Otherwise, the table OTi does not contain the “?” value (line 9) In this case, if OTi is not closed (line 10), an updated table OT is obtained by calling the procedure named make closed(OTi ) (line 11) OT then is put into q (line 12) In the case where the table OTi is closed (line 13), a candidate assumption Ai is generated from OTi (line 14) The candidate assumption Ai is used to check whether it satisfies the two steps of the assume-guarantee rules The step (corresponding to the first assume-guarantee rule) is applied by calling the procedure named Step1(Ai ) to check whether M1 satisfies p in an environment that guarantees Ai by computing the formula Ai M1 p If Step1(Ai ) fails with a counterexample cex (line 15), Ai is too weak for M1 to satisfy p Thus, the candidate assumption Ai must be strengthened by adding a suffix e of cex that witnesses a difference between L(Ai ) and the language of the assumption being learned to Ei of the table OTi (line 16) After that, an updated table OT is obtained by calling the procedure named update(OTi ) (line 17) OT then is put into q (line 18) Otherwise, Step1(Ai ) return true (line 19) This means that Ai is strong enough for M1 to satisfy the property p The step (corresponding to the second assume-guarantee rule) is then applied by calling the procedure named Step2(Ai ) to check that if M2 satisfies Ai by computing the formula true M2 Ai If Step2(Ai ) fails with a counterexample cex (line 20), further analysis is required to identify whether p is indeed violated in M1 M2 or Ai is too strong to be satisfied by M2 Such analysis is based on the counterexample cex If cex witnesses the violation of p in the system M1 M2 (line 21), the algorithm terminates and returns cex (line 22) Otherwise, Ai is too strong to be satisfied by M2 (line 23) The candidate assumption Ai therefore must be weakened by adding a suffix e of cex to Ei of the table OTi (line 24) After that, an updated table OT is obtained by calling the procedure named update(OTi ) (line 25) OT then is put into q (line 26) Otherwise, Step2(Ai ) return true (line 28) This means that the property p holds in the compositional system M1 M2 The algorithm terminates and returns Ai as the minimal assumption (line 29) The Algorithm presents the proposed algorithm for generating the minimal assumption In this algorithm, we 174 algorithm iterates the entire process by looping from line to line 34 until the queue q is empty or a minimal assumption is generated Algorithm Minimized assumption generation Termination and correctness of the proposed algorithm presented in Algorithm have been proven by the Theorem in [7] Output: Am (p) or cex: an assumption Am (p) with a smallest size if M1 M2 satisfies p, and a counterexample cex otherwise Input: M1 , M2 , p: two models M1 and M2 , and a required property p 1: Tool Support 2: 4.1 3: 4: 5: Implementation We have implemented the assumption generation method proposed in [3] (called AG tool) and our proposed minimized assumption generation method presented in Sect (called MAG tool) in a functional programming language named Objective Caml (OCaml) [12] Although the AG tool for L*-based assumption generation method proposed in [3] has been implemented and presented in [4], this tool is not available This means that there is not any available tool which supports the assume-guarantee verification Thus, in order to compare the effectiveness of both methods, we also have to implement the AG tool The AG tool and MAG tool have the same architecture shown in Fig Inputs of these tools are two model M1 and M2 , and a required property p where M1 , M2 , and p are represented by LTSs The AG tool returns an assumption A whereas the MAG tool returns a minimal assumption Am if the CBS M1 M2 satisfies p, and a counterexample cex to show that M1 M2 violates p otherwise For example, Fig shows an applied system of these tools Given two models M1 as the LTS Input and M2 as the LTS Output, and a property as the LTS p, the AG tool returns an assumption A while the MAG tool returns a minimized assumption Am A command line interface of the MAG tool is shown in Fig which corresponds to the generated assumption Am presented in Fig 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: 21: 22: 23: 24: 25: 26: 27: 28: 29: 30: 31: 32: 33: 34: Figure Architecture of the implemented tools 175 Initially, q = empty {q is an empty queue} q.put(OT0 ) {OT0 = (S0 , E0 , T0 ), S0 = E0 = {λ}, where λ is the empty string} while q = empty OTi = q.get() {getting OTi from the top of q} if OTi contains “?” value then for each instance OT of OTi q.put(OT ) {putting OT into q} end for else if OTi is not closed then OT = make closed(OTi ) q.put(OT ) else construct a candidate DFA Ai from the closed OTi if Step1(Ai ) fails with cex then add the suffix e of the counterexample cex to Ei OT = update(OTi ) q.put(OT ) else if Step2(Ai ) fails with cex then if cex witnesses violation of p then return cex else add the suffix e of the counterexample cex to Ei OT = update(OTi ) q.put(OT ) end if else return Ai end if end if end if end if end while Figure An applied example lower cost However, our method has a higher cost for generating the assumption The experiment also implies that our method is potential for verifying large-scale CBS When verifying large-scale CBS, even the behaviors of the software components are very complex, our work only focuses on the observable behaviors of each component With this approach, we hope that our method is potential for verifying practical software In fact, our experiment shown in Table is clear to show the effectiveness of the proposed method because the sizes of GOCS and BS are quite large We did not perform big experiment but we think that this is another issue Moreover, though the proposed method considers the simple case where the CBS only consists of two components M1 and M2 , the experiment presents how to generalize the proposed approach for larger CBS (e.g., GOCS contains five components) We generalize the method for a larger CBS containing n-components M1 , M2 , , Mn (n ≥ 2) by considering the such CBS as a software system which contains two components, i.e., the compositional component M1 M2 Mn−1 and Mn The method for larger CBS consists of the similarly steps as described in Algorithm The implemented tools and the applied systems which are used in our experimental results are available at the site http://www.coltech.vnu.edu.vn/∼hungpn/AGTools/ 4.3 Figure Command line interface of the MAG tool 4.2 Experiment We have applied the proposed method to several systems and compared the method with that proposed in [3] The applied systems are the typical concurrent system Input/Output channel (I/O ver.1) and its evolved versions (I/O ver.2 shown in Fig and I/O ver.3) shown in [5], gas oven control system (GOCS), and banking subsystem (BS) The size (Ass Size), the number of transitions (Trans of Ass.), and the generating time of the generated assumptions are evaluated in this experiment We also evaluate the rechecking time for each system by reusing the generated assumptions for checking the assume-guarantee rules on a PC with Intel(R) Pentium(R)4 CPU 3.20GHz and 512MB RAM Table shows experimental results for the described purpose In the results, the system size is the product of the sizes of the software components and the size of the required property for each CBS Our obtained experimental results imply that the generated minimal assumptions have smaller sizes and number of transitions than the generated ones by the method proposed in [3] These minimal assumptions are effective for rechecking the systems with a Correctness of the Implemented Tools Checking correctness is a big issue for any implemented tool However, we have a nice solution to deal with the problem The correctness of the implemented tools (AG and MAG) means that we have to check whether the assumptions generated by these tools are the actual assumptions by checking that each generated assumption A satisfies the compositional rules (i.e., checking that if A M1 p and true M2 A both hold) For this purpose, we use a tool named LTSA [10] for verifying concurrent systems to check correctness of the generated assumption A by checking the compositional systems A M1 perr and M2 Aerr For each such compositional system, the LTSA tool returns the same result as our verification result for each applied system This means that the correctness of each assumption generated by our tools is ensured by the correctness of the LTSA tool Conclusion We have presented a method and its associated tools for generating minimal assumptions of component-based software verification The method is an improvement of the described L*-based assumption generation method proposed in [3] The key idea of the proposed method is to find a minimal assumption in the search spaces of the candidate 176 System I/O ver I/O Ver I/O Ver GOCS BS Sys Size 18 18 75 180 1200 Ass Size 14 13 Table Experimental results The AG Tool Trans Generating Rechecking Ass of Ass Time (ms) Time (ms) Size 93 7.8 97 9.5 12 94 37.5 110 118 34.5 102 94 33 12 assumptions These minimal assumptions are seen as the environments needed for the components to satisfy a property and for the rest of the system to be satisfied The search space is seen as a search tree where its root is the initial observation table Finding an assumption with a minimal size such that it satisfies the compositional rules thus is considered a search problem in this search tree We have implemented two tools (AG and MAG) for the assumption generation method proposed in [3] and our method proposed in [7] These implementation is used to verify some typical CBS systems to show the effectiveness of the proposed method Our experience so far indicates that the proposed method is potential for verifying practical CBS We are investigating to apply some CBS with their sizes are larger than the sizes of the applied CBS in our experiment to show the practical usefulness of our proposed method Moreover, because the interface of the implemented tools is command line, the generated assumptions are represented in plain text Thus, we are also improving the implemented tools which features graphical display and developing the tools as plugins of the LTSA tool to make our tools more usable Acknowledgments This work is partly supported by the research project No CN 10 03 granted by College of Technology, Vietnam National University, Hanoi References [1] D Angluin Learning Regular Sets from Queries and Counterexamples Information and Computation, 75(2): 87-106, Nov 1987 [2] E M Clarke, O Grumberg, and D Peled Model Checking The MIT Press, 1999 [3] J.M Cobleigh, D Giannakopoulou, and C S Pasareanu Learning assumptions for compositional verification In Proc 9th International Conference on Tools and Algorithms for the The MAG Tool Trans Generating Rechecking of Ass Time (ms) Time (ms) 94 7.6 102 6.3 107 23.5 26 120 24.8 48 109 23.5 Construction and Analysis of Systems (TACAS), pp 331–346, Poland, April 2003 [4] D Giannakopoulou and C S Pasareanu Learning-based assume-guarantee verification In Proc 12th International SPIN Workshop on Model Checking Software, pp 282–287, San Francisco, USA, Aug 2005 [5] P N Hung, T Aoki, and T Katayama Modular conformance testing and assume-guarantee verification for evolving component-based software IEICE Transactions on Fundamentals, E92-A(11): 2772–2780, Nov 2009 [6] P N Hung, T Aoki, and T Katayama An effective framework for assume-guarantee verification of evolving component-based software In Proc of the IWPSE-EVOL Workshops, pp 109–118, ACM, New York, Aug 2009 [7] P N Hung, T Aoki, and T Katayama A minimized assumption Generation method for component-based software verification In Proc 6th International Colloquium on Theoretical Aspects of Computing (ICTAC), LNCS 5684, pp 277-291, Springer-Verlag Berlin Heidelberg, Aug 2009 [8] P N Hung and T Katayama Modular conformance testing and assume-guarantee verification for evolving componentbased software In Proc 15th Asia-Pacific Software Engineering Conference (APSEC), pp 479–486, IEEE Computer Society, Washington, DC, Dec 2008 [9] C B Jones Tentative steps toward a development method for interfering programs ACM Transactions on Programming Languages and Systems (TOPLAS), 5(4): 596–619, Oct 1983 [10] J Magee and J Kramer Concurrency: State Models & Java Programs John Wiley & Sons, 1999 [11] A Nerode Linear automaton transformations Proc of the American Mathematical Society, no 9, pp 541–544, 1958 [12] French national institute for research in computer science and control (INRIA), Objective caml, http://caml.inria.fr/ ocaml/index.en.html, 2004 [13] A Pnueli In transition from global to modular temporal reasoning about programs In Logics and Models of Concurrent Systems, K R.Apt, Ed Nato Asi Series F: Computer And Systems Sciences, Springer-Verlag New York, vol 13, pp 123– 144, 1985 [14] E W Stark A proof technique for rely/guarantee properties In Proc 5th Conference on Foundations of Software Technology and Theoretical Computer Science, pp 369–391, 1985 [15] R L Rivest and R E Schapire Inference of finite automata using homing sequences Information and Computation, 103(2): 299-347, April 1993 177 ... generating time of the generated assumptions are evaluated in this experiment We also evaluate the rechecking time for each system by reusing the generated assumptions for checking the assume-guarantee. .. method and its associated tools for generating minimal assumptions of component-based software verification The method is an improvement of the described L*-based assumption generation method proposed... generated by these tools are the actual assumptions by checking that each generated assumption A satisfies the compositional rules (i.e., checking that if A M1 p and true M2 A both hold) For this