Software testing foundations - testing in the lifecycle

Testing in the Lifecycle Software Testing Foundations 1 Principles 2 Lifecycle 4 Dynamic test techniques 3 Static testing 5 Management 6 Tools 1 Contents Models for testing, economics of testing High level test planning Component Testing Integration testing in the small System testing (non-functional and functional) Integration testing in the large Acceptance testing Maintenance testing Lifecycle 1 2 3 ISEB Foundation Certificate Course 4 5 6 2 V-Model: test levels Integration Testing in the Small Integration Testing in the Large System Testing Component Testing Acceptance Testing Code Design Specification System Specification Project Specification Business Requirements 3 Tests Business Requirements Tests Project Specification Tests System Specification Tests Design Specification Tests Code V-Model: late test design Integration Testing in the Small Integration Testing in the Large System Testing Component Testing Acceptance Testing Design Tests? “We don’t have time to design tests early” 4 TestsTests Business Requirements TestsTests Project Specification TestsTests System Specification TestsTests Design Specification TestsTests Code V-Model: early test design Integration Testing in the Small Integration Testing in the Large System Testing Component Testing Acceptance Testing Run Tests Design Tests 5 Early test design n test design finds faults n faults found early are cheaper to fix n most significant faults found first n faults prevented, not built in n no additional effort, re-schedule test design n changing requirements caused by test design Early test design helps to build quality, Early test design helps to build quality, stops fault multiplication 6 Experience report: Phase 1 Phase 1: Plan 2 mo 2 mo dev test test 150 faults 1st mo. 50 faults users not happy Quality fraught, lots of dev overtime Actual "has to go in" but didn't work 7 Experience report: Phase 2 Source: Simon Barlow & Alan Veitch, Scottish Widows, Feb 96 Phase 2: Plan 2 mo 6 wks dev test test 50 faults 1st mo. 0 faults happy users! Quality smooth, not much for dev to do Actual acc test: full week (vs half day) on time 8 VV&T n Verification •• the process of evaluating a system or component to the process of evaluating a system or component to determine whether the products of the given determine whether the products of the given development phase satisfy the conditions imposed development phase satisfy the conditions imposed at the start of that phase [BS 7925at the start of that phase [BS 7925 1]1] n Validation •• determination of the correctness of the products of determination of the correctness of the products of software development with respect to the user software development with respect to the user needs and requirements [BS 7925needs and requirements [BS 7925 1]1] n Testing •• the process of exercising software to verify that it the process of exercising software to verify that it satisfies specified requirements and to detect faultssatisfies specified requirements and to detect faults 9 Verification, Validation and Testing Verification Validation Testing Any 10 [...]... Certificate Course Contents Models for testing, economics of testing High level test planning Component Testing Integration testing in the small System testing (non-functional and functional) Integration testing in the large Acceptance testing Maintenance testing 30 Component testing n n n n n lowest level tested in isolation most thorough look at detail - error handling - interfaces usually done by programmer... found in testing - calculate cost to fix faults missed by testing Estimate if no data available - your figures will be the best your company has! 20 Lifecycle 1 2 3 4 5 6 ISEB Foundation Certificate Course Contents Models for testing, economics of testing High level test planning Component Testing Integration testing in the small System testing (non-functional and functional) Integration testing in the. .. Deliverables - Test plan - Test design specification - Test case specification - Test procedure specification - Test item transmittal reports - Test logs - Test incident reports - Test summary reports 27 Test Plan 5 n n n 10 Testing tasks - including inter-task dependencies & special skills inter11 Environment - physical, hardware, software, tools - mode of usage, security, office space 12 Responsibilities -. .. issues, providing environment, providing the software to test 28 Test Plan 6 n n n n 13 Staffing and Training Needs 14 Schedule - test milestones in project schedule - item transmittal milestones - additional test milestones (environment ready) - what resources are needed when 15 Risks and Contingencies - contingency plan for each identified risk 16 Approvals - names and when approved 29 Lifecycle 1... Acceptance testing Maintenance testing 21 (Before planning for a set of tests) n n n n n set organisational test strategy identify people to be involved (sponsors, testers, QA, development, support, et al.) examine the requirements or functional specifications (test basis) set up the test organisation and infrastructure defining test deliverables & reporting structure 22 See: Structured Testing, an introduction... level test planning n n What is the purpose of a high level test plan? - Who does it communicate to? - Why is it a good idea to have one? What information should be in a high level test plan? - What is your standard for contents of a test plan? - Have you ever forgotten something important? - What is not included in a test plan? 23 Test Plan 1 n n n 1 Test Plan Identifier 2 Introduction - software items... 2, 1 wk £4000 - fix data (1 day) £350 - pay for fix (3 days maint) £750 - regr test & sign off (2 days) £700 - update doc'n / inform (1 day) £350 - double check + 12% 5 wks £5000 - admin (+7.5%) Totals £800 £1000 £12000 18 Cost of fixing faults 1000 100 10 1 Req Des Test Use 19 How expensive for you? n n Do your own calculation - calculate cost of testing • people’s time, machines, tools - calculate... (4 hrs) £200 - debug ( 5 hr) £25 - test fault fix ( 5 hr) £25 - regression test (8 hrs) £400 £700 £50 16 Hypothetical Cost - 2 Fault Cost Developer User £700 - update doc'n, CM (2 hrs) £100 - update code library (1 hr) £50 - inform users (1 hr) £50 - admin(10% = 2 hrs) £50 £100 Total (20 hrs) £1000 17 Hypothetical Cost - 3 Fault Cost Developer User £1000 £50 (suppose affects only 5 users) - work x 2,... the screen Moves are made by dragging pieces 13 Testing is expensive” n n n Compared to what? What is the cost of NOT testing, or of faults missed that should have been found in test? - Cost to fix faults escalates the later the fault is found - Poor quality software costs more to use • users take more time to understand what to do • users make more mistakes in using it • morale suffers • => lower... module, program testing 31 Component test strategy 1 n n n specify test design techniques and rationale - from Section 3 of the standard* specify criteria for test completion and rationale - from Section 4 of the standard document the degree of independence for test design - component author, another person, from different section, from different organisation, non-human non- *Source: BS 792 5-2 , Software Component . testing High level test planning Component Testing Integration testing in the small System testing (non-functional and functional) Integration testing in the large Acceptance testing Maintenance. Maintenance testing Lifecycle 1 2 3 ISEB Foundation Certificate Course 4 5 6 2 V-Model: test levels Integration Testing in the Small Integration Testing in the Large System Testing Component Testing Acceptance Testing Code Design Specification System Specification Project Specification Business Requirements 3 Tests Business Requirements Tests Project Specification Tests System Specification Tests Design Specification Tests Code V-Model:. test design Integration Testing in the Small Integration Testing in the Large System Testing Component Testing Acceptance Testing Design Tests? “We don’t have time to design tests early” 4 TestsTests Business Requirements TestsTests Project Specification TestsTests System Specification TestsTests Design Specification TestsTests Code V-Model: