Đang tải... (xem toàn văn)
Software Testing Fundamentals (STF) is a platform to gain (or refresh) basic knowledge in the field of Software Testing. If we are to ‘cliche’ it, the site is of the testers, by the testers, and for the testers. Our goal is to build a resourceful repository of Quality Content on Quality. YES, you found it: the notsoultimatebutfairlycomprehensive site for software testing enthusiasts. Since most of our articles are based on various resources, references and experiences, we do not claim any originality or authenticity. Browse the site at your own RISK. Get started by going through Software Testing Basics like Quality Assurance, Quality Control, Software Development Life Cycle and Software Testing Life Cycle. And, level up by understanding Software Testing Levels like Unit Testing, Integration Testing, System Testing and Acceptance Testing. Then, check out the various Software Testing Types like Smoke Testing, Functional Testing, Usability Testing, Security Testing, Performance Testing, Regression Testing and Compliance Testing. Also, learn about Software Testing Artifacts, Defects and Software Testing Metrics. Finally, enjoy our collection of Software Testing Resources like Jokes, Quotes, Jobs, Exercises, Magazines, Blogs and Certifications.
Review Fundamentals of Testing Topic: Fundamentals of Testing Presentation Outline - Software Errors - Software Test Planing - Software Test Requirements - Software Test Design - Review for Software Testing - Software Test Execution - Problem Tracking System - Software Test Management - Software Test Engineers’ Tasks - Test Automation and Tools All Rights Reserved Topic: Fundamentals of Testing Software Errors What is a software error? One common definition of a software error is a mismatch between the program and its specification Definition #1: “A mismatch between the program and its specification is an error in the program if and only if the specification exists and is correct.” Definition #2: “A software error is present for when the program does not what its end user reasonability expects to do.” (Myers, 1976) Definition #3: “There can never be an absolute definition for bugs, nor an absolute determination of their existence The extent to which a program has bugs is measured by the extent to which it fails to be useful This is a fundamentally human measure.” (Besizer, 1984) Topic: Fundamentals of Testing Categories of Software Errors - User interface errors, such as output errors, incorrect user messages - Function errors - Defect hardware - Incorrect program version - Testing errors - Requirements errors - Design errors - Documentation errors - Architecture errors - Module interface errors - Performance errors - Error handling - Boundary-related errors - Logic errors, such as calculation errors - State-based behavior errors - Communication errors - Program structure errors, such as control-flow errors Topic: Fundamentals of Testing Software Test Planing Like other activities in software engineering phases, it is impossible to have a cost-effective software test process without a very good planing, The major objective of software test planing: - generate a well-defined software test plan What content should be included in a software test plan? - Testing activities and schedule - Testing tasks and assignments - Selected test strategy and test models - Test methods and criteria - Required test tools and environment - Problem tracking and reporting - Test cost estimation Other needed items: quality control process and standards Topic: Fundamentals of Testing Software Test Requirements Before starting test design, we must identify our test objectives, focuses, and test items The major purpose is to help us understand what are the targets of software testing This step can be done based on: - Requirements specifications - Inputs from developers - Feedback from customers Benefits are: - Identify and rank the major focus of software testing - Check the requirements to see if they are correct, completed, and testable - Enhance and update system requirements to make sure they are testable - Support the decision on selecting or defining test strategy For example, - for performance testing, we need clear requirements on system performance Topic: Fundamentals of Testing Software Test Requirements The essentials of testing requirements include: - Specified testing methods - Required test types am d test coverage criteria - Selected or required test tools - Testing focuses and test items for each type of software testing An example of performance testing requirements: “Check the system performance to make sure it meet 99% system reliability requirements” A typical example for required test items is: Test item #I: “Test the call waiting feature (REQ #j) during system testing based on the given requirements specifications.” Topic: Fundamentals of Testing Software Test Design Software test design is an important task for software test engineers A good test engineer always know how to come out quality test cases and perform effective tests to uncover as many as bugs in a very tight schedule What you need to come out an effective test set ? - Choose a good test model and an effective testing method - Apply a well-defined test criteria - Generate a cost-effective test set based on the selected test criteria - Write a good test case specification document What is a good test case? - It must have a high probability to discover a software error - It is designed to aim at a specific test requirement - It is generated by following an effective test method - It must be well documented and easily tracked - It is easy to be performed and simple to spot the expected results - It avoids the redundancy of test cases Topic: Fundamentals of Testing Software Test Design What content should be included in a test case? Test Case ID: Wrote By: (tester name) Test Type: Product Name: Test Item: Documented Date: Test Suite#: Release and Version No.: Test case description: Operation procedure: Pre-conditions: Post-conditions: Inputs data and/or events: Expected output data and/or events: Required test scripts: Topic: Fundamentals of Testing Software Test Execution Test execution can be performed: - using manual approach - using a systematic approach - using a semi-automatic approach Basis activities in test execution are: - Select a test case - Set up the pre-conditions for a test case - Set up test data - Run a test case following its procedure - Track the test operations and results - Monitor the post-conditions of a test case & expect the test results - Verify the test results and report the problems if there is any - Record each test execution Testing process input P subset of input oracle test strategy expected output compare P subset of input 30 real output test results What is our goal during testing? • Objective 1: find as many faults as possible • Objective 2: make you feel confident that the software works OK 31 Example constructive approach • Task: test module that sorts an array A[1 n] A contains integers; n 1000 • Solution: take n = 0, 1, 37, 999, 1000 For n = 37, 999, take A as follows: – A contains random integers – A contains increasing integers – A contains decreasing integers • These are equivalence classes: we assume that one element from such a class suffices • This works if the partition is perfect 32 Testing models • Demonstration: make sure the software satisfies the specs • Destruction: try to make the software fail • Evaluation: detect faults in early phases • Prevention: prevent faults in early phases 33 time Testing and the life cycle • requirements engineering – – – – – criteria: completeness, consistency, feasibility, and testability typical errors: missing, wrong, and extra information determine testing strategy generate functional test cases test specification, through reviews and the like • design – functional and structural tests can be devised on the basis of the decomposition – the design itself can be tested (against the requirements) – formal verification techniques – the architecture can be evaluated 34 Testing and the life cycle (cnt’d) • implementation – – – – – check consistency implementation and previous documents code-inspection and code-walkthrough all kinds of functional and structural test techniques extensive tool support formal verification techniques • maintenance – regression testing: either retest all, or a more selective retest 35 Test-Driven Development (TDD) • First write the tests, then the design/implementation • Part of agile approaches like XP • Supported by tools, eg JUnit • Is more than a mere test technique; it subsumes part of the design work 36 Steps of TDD Add a test Run all tests, and see that the system fails Make a small change to make the test work Run all tests again, and see they all run properly Refactor the system to improve its design and remove redundancies 37 (Fagan) inspection • Going through the code, statement by statement • Team with ~4 members, with specific roles: – moderator: organization, chairperson – code author: silent observer – (two) inspectors, readers: paraphrase the code • Uses checklist of well-known faults • Result: list of problems encountered 38 Example checklist • Wrong use of data: variable not initialized, dangling pointer, array index out of bounds, … • Faults in declarations: undeclared variable, variable declared twice, … • Faults in computation: division by zero, mixed-type expressions, wrong operator priorities, … • Faults in relational expressions: incorrect Boolean operator, wrong operator priorities, • Faults in control flow: infinite loops, loops that execute n1 or n+1 times instead of n, 39 Example of control-flow coverage procedure bubble (var a: array [1 n] of integer; n: integer); var i, j: temp: integer; begin for i:= to n if a[i] >= a[i-1] then goto next endif; j:= i; loop: if j = a[j-1] then goto next endif; temp:= a[j]; a[j]:= a[j-1]; a[j-1]:= temp; j:= j-1; goto loop; next: skip; enddo end bubble; input: n=2, a[1] = 5, a[2] = 40 Example of control-flow coverage (cnt’d) procedure bubble (var a: array [1 n] of integer; n: integer); var i, j: temp: integer; begin for i:= to n if a[i] >= a[i-1] then goto next endif; a[i]=a[i-1] j:= i; loop: if j = a[j-1] then goto next endif; temp:= a[j]; a[j]:= a[j-1]; a[j-1]:= temp; j:= j-1; goto loop; next: skip; enddo end bubble; input: n=2, a[1] = 5, a[2] = 41 Control-flow coverage • This example is about All-Nodes coverage, statement coverage • A stronger criterion: All-Edges coverage, branch coverage • Variations exercise all combinations of elementary predicates in a branch condition • Strongest: All-Paths coverage ( exhaustive testing) • Special case: all linear independent paths, the cyclomatic number criterion 42 Mutation testing procedure insert(a, b, n, x); begin bool found:= false; n-1 for i:= to n if a[i] = x then found:= true; goto leave endif enddo; leave: if found then b[i]:= b[i] + else n:= n+1; a[n]:= x; b[n]:= endif end insert; SE, Testing, Hans van Vliet, ©2008 43 Mutation testing (cnt’d) procedure insert(a, b, n, x); begin bool found:= false; n-1 for i:= to n if a[i] = x then found:= true; goto leave endif enddo; leave: if found then b[i]:= b[i] + else n:= n+1; a[n]:= x; b[n]:= endif end insert; SE, Testing, Hans van Vliet, ©2008 44 ...Topic: Fundamentals of Testing Presentation Outline - Software Errors - Software Test Planing - Software Test Requirements - Software Test Design - Review for Software Testing - Software Test... performance testing, we need clear requirements on system performance Topic: Fundamentals of Testing Software Test Requirements The essentials of testing requirements include: - Specified testing. .. Tracking System - Software Test Management - Software Test Engineers’ Tasks - Test Automation and Tools All Rights Reserved Topic: Fundamentals of Testing Software Errors What is a software error?