Test Management

Test Management Software Testing ISEB Foundation Certificate Course 1 Principles 2 Lifecycle 4 Dynamic test techniques 3 Static testing 5 Management 6 Tools 1 Contents Organisation Configuration Management Test estimation, monitoring and control Incident management Standards for testing ISEB Foundation Certificate Course Test Management 1 2 4 5 3 6 2 Importance of Independence Time No. faults Release to End Users 3 Organisational structures for testing n Developer responsibility (only) n Development team responsibility (buddy system) n Tester(s) on the development team n Dedicated team of testers (not developers) n Internal test consultants (advice, review, support, not perform the testing) n Outside organisation (3rd party testers) 4 Testing by developers n Pro’s: know the code bestknow the code best will find problems that the testers will misswill find problems that the testers will miss they can find and fix faults cheaplythey can find and fix faults cheaply n Con’s difficult to destroy own workdifficult to destroy own work tendency to 'see' expected results, not actual resultstendency to 'see' expected results, not actual results subjective assessmentsubjective assessment 5 Testing by development team (buddy) n Pro’s: some independencesome independence technical depthtechnical depth on friendly terms with “buddy” on friendly terms with “buddy” less threateningless threatening n Con’s pressure of own development workpressure of own development work technical view, not business viewtechnical view, not business view lack of testing skilllack of testing skill 6 Tester on development team n Pro’s: independent view of the softwareindependent view of the software dedicated to testing, no development responsibilitydedicated to testing, no development responsibility part of the team, working to same goal: qualitypart of the team, working to same goal: quality n Con’s lack of respectlack of respect lonely, thankless tasklonely, thankless task corruptible (peer pressure)corruptible (peer pressure) a single view / opiniona single view / opinion 7 Independent test team n Pro’s: dedicated team just to do testingdedicated team just to do testing specialist testing expertisespecialist testing expertise testing is more objective & more consistenttesting is more objective & more consistent n Con’s “over the wall” syndrome“over the wall” syndrome may be antagonistic / confrontationalmay be antagonistic / confrontational overover reliance on testers, insufficient testing by reliance on testers, insufficient testing by developersdevelopers 8 Internal test consultants n Pro’s: highly specialist testing expertise, providing support highly specialist testing expertise, providing support and help to improve testing done by alland help to improve testing done by all better planning, estimation & control from a broad better planning, estimation & control from a broad view of testing in the organisationview of testing in the organisation n Con’s someone still has to do the testingsomeone still has to do the testing level of expertise enough?level of expertise enough? needs good “people” skills needs good “people” skills communicationcommunication influence, not authorityinfluence, not authority 9 Outside organisation (3rd party) n Pro’s: highly specialist testing expertise (if outsourced to a highly specialist testing expertise (if outsourced to a good organisation)good organisation) independent of internal politicsindependent of internal politics n Con’s lack of company and product knowledgelack of company and product knowledge expertise gained goes outside the companyexpertise gained goes outside the company expensive?expensive? 10 [...]... test plans test designs test cases: - test input - test data - test scripts - expected results actual results test tools CM is critical for controlled testing What would not be under configuration management? 21 1 2 3 4 5 6 ISEB Foundation Certificate Course Test Management Contents Organisation Configuration Management Test estimation, monitoring and control Incident management Standards for testing... destabilising dependencies - testing is not an independent activity - delivery schedules for testable items missed - test environments are critical Test Iterations (Cycles) - testing should find faults - faults need to be fixed - after fixed, need to retest - how many times does this happen? 24 Test cycles / iterations Theory: Test Iden Des Bld Ex Ver Retest Practice: Test Debug Retest D R D R 3-4 iterations... Certificate Course Test Management Contents Organisation Configuration Management Test estimation, monitoring and control Incident management Standards for testing 14 Problems resulting from poor configuration management n n n n n n n n can’t reproduce a fault reported by a customer can’t roll back to previous subsystem one change overwrites another emergency fault fix needs testing but tests have been... write, the less testing you will be able to do Test Fault analysis & reporting Mike Royce: suspension criteria: when testers spend > 25% time on faults 27 Measuring test execution progress 1 tests planned what would you do? tests run tests passed what does this mean? now release date 28 Diverging S-curve Possible causes poor test entry criteria ran easy tests first Potential control actions tighten... debugging common faults affect all tests stop testing until faults fixed software quality very poor continue testing to scope software quality Note: solutions / actions will impact other things as well, e.g schedules 29 Measuring test execution progress 2 tests planned run passed action taken old release date new release date 30 Measuring test execution progress 3 tests planned run passed action taken... programmer claims it is working OK lots of tests have been run tests have been reviewed / Inspected no faults found in current tests all faults found fixed and retested specified coverage achieved better all tests run after last fault fix, no new faults 35 What actions can you take? n What can you affect? - resource allocation - number of test iterations - tests included in an iteration - entry / exit...Usual choices n n n n Component testing: - done by programmers (or buddy) Integration testing in the small: - poorly defined activity System testing: - often done by independent test team Acceptance testing: - done by users (with technical help) - demonstration for confidence 11 So what we have seen thus far n n n... 21-Dec 09-Feb 32 Control n n Management actions and decisions - affect the process, tasks and people - to meet original or modified plan - to achieve objectives Examples - tighten entry / exit criteria - reallocation of resources Feedback is essential to see the effect of actions and decisions 33 Entry and exit criteria Test Phase 2 "tested" Test Phase 1 is it ready for my testing? Phase 2 Phase 1 Entry... pro's and con's at all levels test techniques offer another dimension to independence (independence of thought) test strategy should use a good mix - "declaration of independence” balance of skills needed 12 Skills needed in testing n n n n n n n Technique specialists Automators Database experts Business skills & understanding Usability expert Test environment expert Test managers 13 1 2 3 4 5 6 ISEB... Review/ Test 19 Status accounting & Configuration Auditing Configuration Identification Configuration Control Status Accounting Configuration Auditing Status Accounting Database Input to SA Database Traceability, impact analysis Procedural Conformance CI Verification Queries and Reports Data Analysis Agree with customer what has been built, tested & delivered 20 Products for CM in testing n n n n n test . view / opinion 7 Independent test team n Pro’s: dedicated team just to do testingdedicated team just to do testing specialist testing expertisespecialist testing expertise testing is more. Tools 1 Contents Organisation Configuration Management Test estimation, monitoring and control Incident management Standards for testing ISEB Foundation Certificate Course Test Management 1 2 4 5 3 6 2 Importance. Test Management Software Testing ISEB Foundation Certificate Course 1 Principles 2 Lifecycle 4 Dynamic test techniques 3 Static testing 5 Management 6 Tools 1 Contents Organisation Configuration