Automatic regression testing for lustre SCADE applications

MINISTRY OF EDUCATION & TRAINING THE UNIVERSITY OF DANANG - - TRINH CONG DUY AUTOMATIC REGRESSION TESTING FOR LUSTRE/SCADE APPLICATIONS ENGINEERING DOCTORAL THESIS Da Nang, 8/2018 MINISTRY OF EDUCATION & TRAINING THE UNIVERSITY OF DANANG - - TRINH CONG DUY AUTOMATIC REGRESSION TESTING FOR LUSTRE/SCADE APPLICATIONS Major: Computer Science Code of Major: 62 48 01 01 ENGINEERING DOCTORAL THESIS Supervisors: Assoc Prof Dr Nguyen Thanh Binh Prof Dr Ioannis Parissis Da Nang, 8/2018 REASSURANCES I hereby certify this thesis done by my work, under the guidance of Assoc Prof Dr Nguyen Thanh Binh and Prof Dr Ioannis Parissis I certify that the research results presented in the thesis are true and are not copied from any other documents All quotations are recorded clearly and completely PhD Student Trinh Cong Duy -i- TABLE OF CONTENTS  REASSURANCES i TABLE OF CONTENTS i ACRONYMS iv LIST OF TABLES v LIST OF FIGURES vi INTRODUCTION 1 Context and Motivation Goals, Objectives and Scope of the Research 3 Contributions of this thesis 4 Thesis structure LUSTRE/SCADE AND REGRESSION TESTING: BASIC CONCEPTS Testing techniques Black-box testing White-box testing Regression testing Introduction Regression Testing Techniques 12 Regression test selection techniques 14 Regression Testing Tools 16 Introduction to Lustre/SCADE 18 Reactive system 18 Synchronous programs 20 Lustre language 21 Specification of a software in Lustre 23 Flows and Clocks in Lustre 25 SCADE environment 27 Structural model in Lustre programs 27 Operator network 27 Paths in an operator 29 -ii- Operator Predicate 29 Activation conditions 30 Testing Lustre/SCADE programs 32 Test data generation 32 Coverage criteria for Lustre programs 34 Conclusion 36 USING MODEL CHECKER FOR TESTING LUSTRE/SCADE PROGRAMS 37 Model checking technique 37 Introduction to model checking 37 Kripke structure 39 Temporal logics 41 Testing with model checkers 43 Lesar: Tool for model checking of Lustre/SCADE programs 46 A solution to generate test data for Lustre/SCADE programs 49 Using model checker in test cases generation for Lustre/SCADE 49 The AGTC algorithm 52 Case study 54 Conclusion 57 REGRESSION TESTING APPROACH FOR LUSTRE/SCADE PROGRAMS 59 Motivation 59 Research scope in regression testing for Lustre/SCADE programs 63 Proposed approaches for regression testing 64 The GSRS approach 66 The GSCR approach 72 The GOPN approach 78 Conclusion 85 LUSREGTES: A REGRESSION TESTING TOOL FOR LUSTRE/SCADE PROGRAMS 86 Introduction 86 Test Execution Environment 86 The LUSREGTES tool 89 Case studies 90 -iii- Heater Controller System 90 “U-turn” section management system 99 Evaluation summary 103 Conclusion 104 CONCLUSIONS AND FUTURE WORKS 105 PUBLICATIONS 107 REFERENCES 108 -iv- ACRONYMS API Application Programming Interface AUT Application Under Test CARE Complementarity, Assignment, Redundancy and Equivalence CASE Concurrent, Alternate, Synergistic and Exclusive SCADE Safety-Critical Application Development Environment DFA Deterministic Finite State Automaton FSM Finite State Machines MBT Model-based testing NFA Nondeterministic Finite State Automaton PFSM Probabilities Finite State Machine RBT Requirement-based testing SQL Structured Query Language UML Unified Modeling Language SRS Software Requirements Specification SCR Software Cost Reduction GSRS Generation of test cases in regression test using SRS GSCR Generation of test cases in regression test using SCR GOPN Generation of test cases in regression test using Operator Network -v- LIST OF TABLES Table 1.1 Output for never program 23 Table 1.2 The use of the operators when and current [4] 26 Table 1.3 The examples of paths 29 Table 1.4 Activation conditions for all Lustre operators 31 Table 1.5 The activation condition from paths 32 Table 2.1 Summary of LTL operators .42 Table 2.2 Summary of CTL operators .43 Table 2.3 Paths of latch program .55 Table 2.4 Paths and the correspoding activation conditions 56 Table 2.5 The generated test cases based on activation conditions 56 Table 3.1 The test suite for version of the UMS 70 Table 3.2 Status of test suite in version of the UMS 71 Table 3.3 SCR Event Table for mcPressure 76 Table 3.4 Modified SCR Event Table for mcPressure 77 Table 3.5 The difference between the two SCR tables 77 Table 3.6 Comparing the GSRS, GSCR and GOPN approaches 83 Table 4.1 The Lustre program of version .91 Table 4.2 The Lustre program of version .92 Table 4.3 Paths and Activation conditions of version 93 Table 4.4 Test data for version 95 Table 4.5 The set of paths P2 in new version 97 Table 4.6 Set of test data T’’ for regression testing 98 Table 4.7 Paths and Activation conditions of version 100 Table 4.8 Comparingthe LUSREGTES tool with other tools .104 -vi- LIST OF FIGURES Figure 1.1 Regression testing in software maintenance process [9] 12 Figure 1.2 Regression Testing Techniques 13 Figure 1.3 A reactive system [29] 19 Figure 1.4 Synchronous program [31] .20 Figure 1.5 Example of a Lustre program 23 Figure 1.6 Illustrating a SCADE model and Lustre program [6] 27 Figure 1.7 Operator network of the “never” node 28 Figure 2.1 Software Formal Verification Process 38 Figure 2.2 A typical model checking work-flow [43] .38 Figure 2.3 An example of Kripke structure [45] 40 Figure 2.4 Creating test cases with model checker 45 Figure 2.5 SMV Language Example for Train door controller .45 Figure 2.6 The counter-example for Train Door Controller 46 Figure 2.7 An example of Lesar input .48 Figure 2.8 Model checking for Lustre program 51 Figure 2.9 Model of test case generator for Lustre program using activation condition 51 Figure 2.10 Using Lesar to generate test cases 52 Figure 2.11 The AGTC algorithm .53 Figure 2.12 A Lustre program implementing a latch .54 Figure 2.13 The operator network for latch program 55 Figure 3.1 Regression testing in a Lustre program development process .61 Figure 3.2 Evolution of the Lustre program and regression test 63 Figure 3.3 Test case life cycle in regression testing 64 Figure 3.4 Process to determine the status of the test cases .66 Figure 3.5 Generation of test cases for regression testing .67 Figure 3.6 The UMS system and its environment [4] 68 -vii- Figure 3.7 Overall approach for regression testing using SCR .74 Figure 3.8 Two versions of SIS 76 Figure 3.9 The overall approach of regression testing for Lustre programs 79 Figure 3.10 Generating and selecting test cases for version 79 Figure 3.11 Generating and selecting test cases for version 81 Figure 3.12 The AGTR algorithm .82 Figure 4.1 The REGTESLUS Testing Environment .86 Figure 4.2 Module 87 Figure 4.3 The method get the list of paths and respectively activation conditions 87 Figure 4.4 Module 88 Figure 4.5 Algorithm for comparing two sets of path 88 Figure 4.6 The REGTESLUS screenshots .89 Figure 4.7 The Heater Controller System architecture [37] 90 Figure 4.8 The operator network for version 92 Figure 4.9 Paths and Activation Conditions in LUSREGTES 94 Figure 4.10 The operator network for version 96 Figure 4.11 The result of the comparison 96 Figure 4.12 Chart of test cases in regression test .98 Figure 4.13 The Lustre program for the UMS system .99 Figure 4.14 Paths and Activation Conditions in LUSREGTES 100 Figure 4.15 The result of the comparison 102 Figure 4.16 Set of test data for regression testing of the UMS system 102 -102- (ack_BC ,L4 ,do_BC) PA_7 = (true and (not(L4) or only_on_B))-> (true and (not(L4) or only_on_B) or pre(PA_7)); AC_7 = (if PA_7 then else 0) + AC_6; tel; Continue, the tool will compare two set of paths P1 and P2, then classifies the paths into three subsets PA, PB and PC matching with Path A, Path B and Path C in Figure 4.15 Figure 4.15 The result of the comparison The test cases in Figure 4.16 are used for regression testing of version This allows testing only the affected parts and new parts in version Figure 4.16 Set of test data for regression testing of the UMS system -103- Analyzing the test cases used for version and the test cases used for version 2, we observe that: − 22 % of test cases have been removed; − 41 % of test cases have been reused; − 37 % of test cases have been newly created Evaluation summary Heater Controller System and U-turn Section Management System are two typical examples of reactive systems The LUSREGTES tool was used for generating test data for version and for generating test data for regression testing in version The test data generation process is performed automatically using the source code of the Lustre program Rather than having to re-run all test cases for the new version, the tool has eliminated many unnecessary test cases, and so reduced the cost of testing (37,5% for Heater Controller System and 63% for U-turn Section Management System) It also provided a full set of test cases All test cases created by the tool cover all the paths on the operator network of Lustre/SCADE program In brief, the LUSREGTES tool illustrates that our approach has met the requirements set out in the research objectives: − Creating test data for the Lustre program from the source code; − Selecting and generating test data for regression testing; − The test data satisfying the data flow coverage criterion; − Providing useful data for future reference Finally, when compared with the Lutess tool and the GATel tool presented in section 1.5.1, the LUSREGTES tool brings a significant improvement, namely the function of generating test cases for regression testing of Lustre/SCADE programs, as shown in Table 4.8 -104- Table 4.8 Comparingthe LUSREGTES tool with other tools Tool Lutess Comparative criteria Technique GATeL LUSREGTES Black-box White-box White-box Test data generation method Random Translating to Prolog Use direct: Paths and ACs Coverage criterion Control flow Data flow Data flow Regression testing No support No support Support Cost Medium Medium Low Easy to use Easy Not easy Easy Conclusion In this chapter, we have presented the LUSREGTES tool based on the previously proposed approaches The tool automatically creates test data for regression testing of Lustre/SCADE programs, identifies test data which should be removed and re-used Commonly, in the development process, the number of test cases raises after each change and becomes larger and larger which leads to increase the costs of regression test This tool helps removing the unnecessary test data; therefore, it saves considerable costs for regression testing process Part of this research was published in the journal “International Journal of Electrical and Computer Engineering“4 Nguyen Thanh Bình, Trinh Cong Duy, Ioannis Parissis, “LusRegTes: A Regression Testing Tool for Lustre Programs,” International Journal of Electrical and Computer Engineering (IJECE, ISSN: 2088-8708, a SCOPUS indexed Journal), Vol 7, No 5, 2017 -105- CONCLUSIONS AND FUTURE WORKS Compared to the objectives, the thesis has achieved the results as follows: − Studying the basic issues in regression testing and regression testing techniques, applications of regression testing, the state of the art in regression testing − Studying and analysing basic features of reactive systems, synchronous approach, the Lustre language and SCADE environment; identifing the structural coverage of Lustre programs; defining the activation conditions of paths on operator network of the equivalent Lustre programs − Figuring out the overview of model checker, using model checking in software testing Instead of using modeling languages to build models of Lustre programs and LTL properties that define trap properties, we have suggested the solution which uses activation conditions on the operator network from Lustre programs combined with model checker on main source of Lustre program to create test data for Lustre/SCADE programs This solution helps to remove the manual input definition from the model checking, as well as save costs since this solution could be fully automated in order to reduce costs and increase the efficiency of testing process of Lustre/SCADE programs − On the basis of analysing the characteristics of Lustre/SCADE programs, the thesis proposes to fully generate test cases for regression testing of Lustre/SCADE programs When the changes appear upon Lustre/SCADE program, we identify which data have just been removed, other data are re-used from the old version and new test data need be created We proposed and experimented three approaches: GSRS - Generation of test cases in regression test using software requirements in natural language; GSCR - Generation of test cases in regression test using SCR; GOPN - Generation of test cases in regression test using operator network Most importantly, we can propose the best suited solution to automate the process of generating test data for regression test based on special characteristics of Lustre -106- programs which are operator network, paths on the operator network and activation conditions − A regression testing tool has been developped implementing the GOPN approach, called LUSREGTES The tool automatically creates test data for regression testing of Lustre/SCADE programs, identifies test data which should be removed and re-used Commonly, in the development process, the number of test cases raises after each change and becomes larger and larger which leads to increase in the costs of regression test This tool helps removing the unnecessary test data; therefore it saves considerable time and effort for regression testing process − All research results and source code of LUSREGTES tool are publicly available on the website: https://sites.google.com/view/lusregtes Everyone can refer or continue to develop this reseach In addition to the obtained results in the thesis, some issues may be studied in the future: − We intend to support the construction of reports for test management with test cost, test coverage, and defects found − The results obtained by the experimental evaluation showed that in many cases, one test suite satisfies several activation conditions It would be interesting to investigate more thoroughly the coverage ratio that the criteria yield with regard to the number of the required test cases and hence explore ways of finding a minimal set of test cases set that satisfy all the activation conditions for a given criterion − To demonstrate scalability of the method and applicability to more realistic, complicated software systems, we plan to use larger specifications taken from real life examples − Currently, there are not many similar studies in automated regression testing for Lustre/Scade programs, so evaluating our solutions and other solutions is difficult − The LUSREGTES tool developed in this work can be further extended in the future We will study more additional new functionality and integrate the existing tool -107- PUBLICATIONS [1] Trinh Cong Duy, Nguyen Thanh Bình, Ioannis Parissis, “Automatic Generation of Test Cases in Regression Testing for Lustre/SCADE Programs,” Journal of Software Engineering and Applications (JSEA, ISSN: 1945-3124), Pages: 27-35, Vol.6, No.10A, 2013 [2] Trịnh Công Duy, Nguyễn Thanh Bình, Ioannis Parissis, “Sinh ca kiểm thử tự động kiểm thử hồi quy cho hệ thống phản ứng,” Kỷ yếu Hội nghị Quốc gia lần thứ VI Nghiên cứu ứng dụng Công nghệ thông tin (FAIR 2013), Trang: 484-494, 2013 [3] Trinh Cong Duy, Nguyen Thanh Bình, “Applying model checking for regression testing based on requirements specifica,” 13th International Symposium on Advanced Technology - ISAT13 - Engineering innovation for sustainable future, Pages: 61-67, 2014 [4] Trịnh Cơng Duy, Nguyễn Thanh Bình, ”Sinh ca kiểm thử cho hệ thống phản ứng sử dụng cơng cụ kiểm chứng mơ hình NUSMV,” Chun san Kỹ tḥt và Cơng nghệ, Tạp chí khoa học Đại Học Huế, Trang 55-66, tập 106, Số 7, 2015 [5] Trịnh Cơng Duy, Nguyễn Thanh Bình, “Xây dựng cơng cụ sinh liệu thử cho chương trình Lustre/SCADE dựa kiểm chứng mơ hình,” Tạp chí khoa học cơng nghệ Đại học Đà Nẵng, Trang 84-90, Số 9(94), 2015 [6] Trịnh Cơng Duy, Nguyễn Thanh Bình, Ioannis Parissis, “Sinh liệu thử cho ứng dụng Lustre/SCADE sử dụng điều kiện kích hoạt,” Kỷ yếu Hội nghị Quốc gia lần thứ VIII Nghiên cứu ứng dụng Công nghệ thông tin (FAIR 2015), Trang: 628-639, 2015 [7] Trinh Cong Duy, Nguyen Thanh Binh, Ioannis Parissis, “A regression testing approach for Lustre/SCADE programs,” The Sixth International Symposium on Information and Communication Technology (SoICT 2015), Hue City, ACM Publisher, 2015 [8] Nguyen Thanh Binh, Trinh Cong Duy, Ioannis Parissis, “LusRegTes: A Regression Testing Tool for Lustre Programs,“ International Journal of Electrical and Computer Engineering (IJECE, ISSN: 2088-8708, a SCOPUS indexed Journal),Vol 7, No 5, 2017 -108- REFERENCES [1] David Parnas, John van Schouwen and Shu Po Kwan, "Evaluation of SafetyCritical Software," Communications of the ACM, vol 33(6), pp 636-648, 1990 [2] F Boussinot and R De Simone, "The Esterel language," Proceedings of the IEEE, vol 1293–1304, p 79(9), 1991 [3] P Le Guernic, T Gautier, M Le Borgne and C Le, "Programming Real-Time Applications with SIGNAL," Proceedings of the IEEE, vol 79(9), p 1321–1336, 1991 [4] N Halbwachs, F Lagnier and C Ratel, "Programming and verifying real-time systems by means of the synchronous data-flow programming language Lustre," IEEE Transactions on Software Engineering, Special Issue on the Specification and Analysis of Rea Time Systems, 1992 [5] Albert Benveniste, Paul Caspi, Stephen A Edwards, Nicolas Halbwachs, Paul Le Guernic and Robert de Simone, "The synchronous languages 12 years later," Proceedings of the IEEE, vol 91(1), p 64–83, 2003 [6] Michel Labrousse - Esterel Technologies SA, "Scade language reference manual," Technical report SC-LRM, vol SC/70257, 2005 [7] RTCA, Inc., Do-178b, "Software considerations in airborne systems and equipment certification," Technical report, 1992 [8] B Hetzel, "The Complete Guide to Software Testing," John Wiley and Sons, 1988 [9] Biswas, Swarnendu and et al, "Regression test selection techniques: A survey," Informatica, 2011 [10] S Beydeda and V Gruhn, "Integrating white- and black-box techniques for class-level testing object-oriented prototypes," Software Engineering and Applications Conference, p 23–28, 2000 [11] H K N Leung and L J White, "A Cost Model to Compare Regression Test Strategies," Proc Conference on Software Maintenance, pp 201-208, 1991 -109- [12] Y Chen, R L Probert and D.P Sims, "Specification based Regression test selection with risk analysis," Proceeding of the Conference of the center for advance studies on collaborative research, 2002 [13] T L Graves, M.J Harrold, J Kim, A Porter and G Rothermel, "An Empirical Study of Regression Test Selection Techniques," ACM Transactions on Software Engineering and Methodology, Vols 10, No 2, pp 184-208, 2001 [14] G Rothermel and M.J Harrold, "Analyzing Regression Test Selection Techniques," IEEE Transactions on Software Engineering, Vols 22, No.8, pp 529-551, 1996 [15] G Rothermet, S Elbaum, A Malishevsky and P Kallak, "The Impact of Test Suite Granularity on the Cost Effectiveness of Regression Testing," Proceedings of the 24th International Conference on Software Engineering, pp 130-140, 2002 [16] X Lin, "Regression Testing in Research And Practice," Computer Science and Engineering, Vols Department University of Nebraska, Lincoln [17] Huy Vu Do, Ajitha Rajan, Lydie du Bousquet, Laya Madani, Michel Delaunay and Ioannis Parissis, "Survey of Regression Test Selection Techniques," 210 [18] P Benedusi, A Cimitile and U De Carlini, "Post-Maintenance Testing Based on Path Change Analysis," Proceedings of the Conference on Software Maintenance, pp 352-361, 1988 [19] R Gupta, M.J Harrold and M.L Soffa, "An approach to regression testing using slicing," Proceedings of the Conference on Software Maintenance, pp 299-308, 1992 [20] M.J Harrold and M.L Soffa, "An Incremental Approach to Unit Testing during Maintenance," Proceedings of the Conference on Software Maintenance, pp 362-367, 1988 [21] S Baes and S Horwitz, "Incremental Program Testing Using Program Dependence Graphs," Proceedings of the 20th ACM Symposium on Principles of programming languages, pp 384-396, 1993 -110- [22] D Binkley, "Reducing the cost of Regression Testing by Semantics Guided Test Case Selection," Proceedings of the Conference on Software Maintenance, pp 251-260, 1995 [23] J Laski and W Szermer, "Identification of Program Modifications and Its Applications in Software Maintenance," Proceedings of the Conference on Software Maintenance, pp 282-290, 1992 [24] H Agrawal, J Horgan, E Krauser and S London, "Incremental Regression Testing," Proceedings of the Conference on Software Maintenance, pp 348-357, 1993 [25] G Rothermel and M.J Harrold, "A Safe, Efficient Algorithm for Regression Test Selection," Proceedings of the Conference on Software Maintenance, pp 358-367, 1993 [26] G Rothermel and M.J Harrold, "Selecting Tests and Identifying Test Coverage Requirements for Modified Software," ACM SIGSOFT international symposium on Software testing and analysis, pp 169-184, 1994 [27] L Xu, M Dias and D Richardson, "Generating Regression Tests via Model Checking," Proceedings of the 28th Annual International Computer Software and Applications Conference (COMPSAC ’04), pp 336-341, 2004 [28] "http://www.adventnet.com/products/qengine/index.html" [29] D Harel and A Pnueli, "On the development of reactive systems," SpringerVerlag New York, p 477–498, 1985 [30] Pascal Raymond, Xavier Nicollin, Nicolas Halbwachs and Daniel Weber, "Automatic testing of reactive systems," IEEE Real-Time Systems Symposium, p 200–209, 1998 [31] Benveniste and G Berry, "The Synchronous Approach to Reactive and RealTime Systems," Proceedings of the IEEE, vol 79(9), p 1270–1282, 1991 [32] N Halbwachs, P Caspi, P Raymond and D Pilaud, "The synchronous data flow programming language Lustre," Proceedings of the IEEE, vol 79(9), p 1305–1320, 1991 -111- [33] Paul Caspi, Daniel Pilaud, Nicolas Halbwachs and John Plaice, "Lustre: A declarative language for programming synchronous systems," POPL, p 178–188, 1987 [34] Ioannis Parissis, "A Formal Approach to Testing Lustre Specifications," in Proceedings of the 1st International Conference on Formal Engineering Methods (ICFEM '97) IEEE Computer Society, 1997 [35] V Papailiopoulou, L Madani, L du Bousquet and I Parissis, "Extending structural test coverage criteria for lustre programs with multi-clock operators," ”, in The 13th International Workshop on Formal Methods for Industrial Critical Systems (FMICS), 2008 [36] Esterel Technologies , "Introdution of Products and solutions," SCADE Suite, [Online] Available: http://www.esterel-technologies.com/products/scade- suite [Accessed 20 10 2016] [37] A Lakehal and I Parissis, "Lustructu: A tool for the automatic coverage assessment of lustre programs," in proceedings of the 16th IEEE International Symposium on Software Reliability Engineering (ISSRE 2005), Updated and Available from: Ioannis Parissis (May 29, 2014), p 301–310, 2014 [38] Lakehal, Abdesselam and Ioannis Parissis, "Structural coverage criteria for LUSTRE/SCADE programs," Software Testing, Verification and Reliability, vol 19.2, pp 133-154, 2009 [39] Virginia Papailiopoulou, "Automatic testing of Lustre/SCADE programs," Computer Science, Université Joseph Fourier Grenoble I, 2010 [40] Lydie du Bousquet and Nicolas Zuanon, "An overview of lutess: A specificationbased tool for testing synchronous software," in ASE, p 208–215, 1999 [41] Bruno Marre and Benjamin Blanc, "Test selection strategies for lustre descriptions in Gatel," Electr Notes Theor Comput Sci., p 93–111, 2005 [42] Ajitha Rajan, "Coverage metrics for requirements-based testing," Phd thesis, University of Minnesota, Minneapolis, 2008 [43] Gordon Fraser, Franz Wotawa and Paul E Ammann, "Testing with model checkers: a survey," Software Test Verif Reliab, 2009 -112- [44] Mark Utting, Alexander Pretschner and Bruno Legea, "A taxonomy of modelbased testing," Technical Report, Department of Computer Science, The University of Waikato, vol 04/2006, 2006 [45] Browne, M.C , E M Clarke and Grumberg, "Characterizing Kripke Structures in Temporal Logic," Carnegie Mellon University, Computer Science Department, 1987 [46] Amir Pnueli, "The temporal logic of programs," in 18th Annual Symposium on Foundations of Computer Science, p 46–57, 1977 [47] Edmund M Clarke and E Allen Emerson, "Design and synthesis of synchronization skeletons using branching-time temporal logic," Logic of Programs, Workshop, Springer-Verlag., p 52–71, 1982 [48] A F E Belinfante, L Frantzen and C Schallhar, "Tools for test case generation," Lecture Notes in Computer Science, Springer Verlag, p 391– 438, 2005 [49] Andrea Arcuri, Muhammad Zohaib Iqbal and Lionel B, "Black-box system testing of real-time embedded systems using random and search based testing," in Proceedings of the 22nd IFIP WG 6.1 International Conference on Testing Software and Systems, ICTSS’10, Berlin, Heidelberg, 2010 [50] Prasad Ramesh Bokil, "Generating Effective Test Suites for Reactive Systems using Specification Mining," Faculty of Business Bond University, Australia, 2014 [51] A Gargantini and C Heitmeyer, "Using Model Checking to Generate Tests from Requirements Specifications," Proc., Joint 7th Eur Software Engineering Conf and 7th ACM SIGSOFT Intern Symp on Foundations of Software Eng (ESEC/FSE99), 1999 [52] P E Ammann, P E Black and W Majurski, "Using Model Checking to Generate Tests from Specifications," Proceedings of 2nd ICFEM’98, IEEE, 1998 [53] P Ammann, Paul E Black and W Ding, "Model Checkers in Software Testing," National Institute of Standards and Technology, NIST-IR 6777, 2002 [54] J Callahan, F Schneider and S Easterbrook,, "Automated software testing using model checking," Proceedings 1996 SPIN workshop, Rutgers, 1996 -113- [55] A Engels, L Feijis and S Mauw, "Test Generation for Intelligent Networks Using Model Checking," Proc TACSS’97, 1997 [56] P Krishnan, "Uniform Descriptions for Model Based Testing," Australian Software Engineering Conference , 2004 [57] Tanja E Vos, Felix F Lindlar, Benjamin Wilmes, Arthur I Baars, Peter M Kruse, Hamilton Gross and Joachim Wegener, "Evolutionary functional blackbox testing in an industrial setting," Software Quality Journal, vol 21, no 2, pp 259-288, 2013 [58] JeeHyun Hwang, Tao Xie, Donia El Kateb, Tejeddine and Yves Le Traon, "Selection of regression system tests for security policy evolution," in Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering, New York, NY, USA, 2012 [59] Kalpesh Kapoor and Jonathan Bowen, "Experimental Evaluation of the Variation in Effectiveness for DC, FPC and MC/DC Test Criteria," in Proceedings of the 2003 International Symposium on Empirical Software, IEEE Computer Society, 2003 [60] Lihua Xu, Marcio Dias and Debra Richardson, "Generating regression tests via model checking," Proceedings of the 28th Annual International Computer Software and Applications Conference (COMPSAC’04), p 336–341, 2004 [61] Gordon Fraser, Bernhard Aichernig and Franz Wotaw, "Handling model changes: Regression testing and test-suite update with model-checkers," Electronic Notes in Theoretical Computer Science, vol 190, p 33–46, 2007 [62] Heitmeyer, C L and J J Marciniak, "Software Cost Reduction," Encyclopedia of Software Engineering, vol 2, 2002 [63] K Heninger, D Parnas, J Shore and J Kallander, "Software requirements for the A-7E aircraft," Technical Report 3876, Naval Research Lab, 1978 [64] S R Faulk, L Finneran, J Kirby, Jr., S Shah and J Sutton, "Experience applying the CoRE method to the Lockheed C-130J," Proc 9th Annual Conf on Computer Assurance (COMPASS ’94), 1994 -114- [65] C L Heitmeyer , R D Jeffords and B G Labaw, "Automated consistency checking of requirements specifications," ACM Trans on Software Eng and Methodology, 1996 [66] Glenford J Myers and Corey Sandler, "The Art of Software Testing," John Wiley & Sons, 2004 [67] W.E Perry, "A Standard for Testing Application Software," 1990 [68] Richard G Hamlet, "Theoretical comparison of testing methods," Symposium on Testing, Analysis, and Verification, p 28–37, 1989 [69] Joe W Duran and Simeon Ntafos, "A report on random testing," ICSE ’81: Proceedings of the 5th international conference on Software engineering, IEEE Press, p 179–183, 1981 [70] S Rao Kosaraju, "Analysis of structured programs," Journal of Computer and System Sciences, vol 9(3), p 232–255, 1974 [71] Thomas J McCabe, "A complexity measure," IEEE Transactions on Software Engineering, vol 2(4), p 308–320, 1976 [72] L White and E Cohen, "A domain strategy for computer program testing," IEEE Transactions on Software Engineering, vol 6(3), p 247–257, 1980 [73] J J Chilenski and S P Miller, "Applicability of modified condition/decision coverage to software testing," IEEE Transactions on Software Engineering, p 193–200, 1994 [74] Hayhurst Kelly J, Veerhusen Dan S, Chilenski John J and Rierson Leanna K, "A practical tutorial on modified condition/decision coverage," Technical report, 2001 [75] M R Woodward, D Hedley and M A Hennell, "Experience with path analysis and testing of programs," IEEE Trans Softw Eng, p 278–286, 1980 [76] Sandra Rapps and Elaine J Weyuker, "Selecting software test data using data flow information," IEEE Trans Software Eng, p 367–375, 1985 -115- [77] Phyllis G Frankl and Elaine J Weyuker, "An applicable family of data flow testing criteria," IEEE Trans Software Eng, p 1483–1498, 1988 [78] Lori A Clarke, Andy Podgurski, Debra J Richardso and Steven J Zeil, "A formal evaluation of data flow path selection criteria," IEEE Trans Software Eng, p 1318–1332, 1989 [79] Simeon C Ntafos, "An evaluation of required element testing strategies," International Conference on Software Engineering, p 250–256, 1984 [80] Sandra Rapps and Elaine J Weyuker, "Selecting software test data using data flow information," IEEE Trans Software Eng, p 367–375, 1985 [81] Janusz W Laski and Bogdan Korel, "A data flow oriented program testing strategy," IEEE Transactions on Software Engineering, p 347–354, 1983 [82] A Lakehal and I Parissis, "Lustructu: A tool for the automatic coverage assessment of lustre programs," in proceedings of the 16th IEEE International Symposium on Software Reliability Engineering (ISSRE 2005), p 301–310, 2005 [83] Besnik Seljimi and Ioannis Parissis, "Automatic generation of test data generators for synchronous programs: Lutess v2," DOSTA ’07: Workshop on Domain specific approaches to software test automation, no Dubrovnik, Croatia, ACM., p 8–12, 2007 [84] F Ouabdesselam and I Parissis, "Testing synchronous critical software," In 5th International Symposium on Software Reliability Engineering, 1994 [85] Matthew Hennessy and Robin Milner, "Algebraic laws for nondeterminism and concurrency," J ACM, :, doi: 10.1145/2455.2460, 198, vol 32(1), p 137–161, 1985 [86] Dexter Kozen, "Results on the propositional mu-calculus," Theor Comput Sci., vol 27, p 333–354, 1983 [87] Alessandro Cimatti, Edmund M Clarke, Fausto Giunc and Marco Roveri, "NUSMV: A New Symbolic Model Verifier," CAV ’99: Proceedings of the 11th International Conference on Computer Aided Verification, SpringerVerlag, p 495–499, 1999 -116- [88] Gerard J Holzmann, "The Model Checker SPIN," IEEE Trans Softw Eng., vol 23(5), p 279–295, 1997 [89] Leonardo de Moura, Sam Owre, Harald Rueß, John Rushby, N Shankar, Maria Sorea and Ashish Tiwari, "SAL 2," Computer-Aided Verification, CAV 2004, Lecture Notes in Computer Science, SpringerVerlag, , vol 3114 , pp 496–500, 2004 ... real-time systems developped in the Lustre/ SCADE language More precisely, the topic "Automatic regression testing for Lustre/ SCADE applications" has been selected for this thesis to contribute to... this thesis is to figure out automatic regression testing techniques On that basis, we propose a solution for generating test data in regression testing for Lustre/ SCADE applications As a result,... solve our problems Chapter LUSREGTES: a regression testing tool for Lustre/ SCADE programs Chapter presents the LUSREGTES tool for regression testing Lustre/ SCADE programs implementing and illustrating