Functional testing potx

Functional testing 2 Contents  Introduction to functional testing  Functional testing techniques  Boundary Value testing  Equivalence Class testing  Special Value testing  Decision Tables 3 Introduction to functional testing 4 Functional testing  Functional testing: Deriving test cases from program specifications  Functional refers to the source of information used in test case design, not to what is tested  Also known as:  specification-based testing (from specifications)  black-box testing (no view of the code)  Functional specification = description of intended program behavior  either formal or informal 5 Systematic vs Random Testing  Random (uniform):  Pick possible inputs uniformly  Avoids designer bias  A real problem: The test designer can make the same logical mistakes and bad assumptions as the program designer (especially if they are the same person)  But treats all inputs as equally valuable  Systematic (non-uniform):  Try to select inputs that are especially valuable  Usually by choosing representatives of classes that are apt to fail often or not at all  Functional testing is systematic testing 6 Why Not Random?  Non-uniform distribution of faults  Example: Java class ―roots‖ applies quadratic equation Incomplete implementation logic: Program does not properly handle the case in which b 2 - 4ac =0 and a=0 Random sampling is unlikely to choose a=0.0 and b=0.0 7 Systematic Partition Testing Failure (valuable test case) No failure Failures are sparse in the space of possible inputs but dense in some parts of the space If we systematically test some cases from each part, we will include the dense parts Functional testing is one way of drawing pink lines to isolate regions with likely failures The space of possible input values (the haystack) 8 Functional testing: exploiting the specification  Functional testing uses the specification (formal or informal) to partition the input space  E.g., specification of ―roots‖ program suggests division between cases with zero, one, and two real roots  Test each category, and boundaries between categories  No guarantees, but experience suggests failures often lie at the boundaries (as in the ―roots‖ program) 9 Why functional testing?  Timely  Often useful in refining specifications and assessing testability before code is written  Effective  finds some classes of fault (e.g., missing logic) that can elude other approaches  Widely applicable  to any description of program behavior serving as spec  at any level of granularity from module to system testing.  Economical  typically less expensive to design and execute than structural (code-based) test cases 10 Early functional test design  Program code is not necessary  Only a description of intended behavior is needed  Even incomplete and informal specifications can be used  Although precise, complete specifications lead to better test suites  Early functional test design has side benefits  Often reveals ambiguities and inconsistency in spec  Useful for assessing testability  And improving test schedule and budget by improving spec  Useful explanation of specification  or in the extreme case (as in XP), test cases are the spec [...].. .Functional versus Structural: Classes of faults  Different testing strategies (functional, structural) are most effective for different classes of faults  Functional testing is best for finding design faults  Structural testing is best for finding programming faults 11 Functional vs structural test: granularity levels  Functional test applies at all granularity... two (or more) faults.‖ 26 Extensions of Boundary Testing  Boundary value testing   Robustness testing   Exceed the limits Worst-case testing   Stay within the limits Like boundary value testing above, but discard Single Fault Assumption Robust worst-case testing  Like robustness testing above, but discard Single Fault Assumption 27 Boundary Value Testing The basic idea of boundary value analysis... of boundary values (0 cities, 6 characters) and error cases Empty; 1-4 characters; 6 characters; very long Non-digit characters Non-character data 16 Functional testing techniques     Boundary Value testing Equivalence Class testing Special Value testing Decision Tables 17 Examples 18 The Triangle Program In order for 3 integers a, b, and c to be the sides of a triangle, we must have c1 a + b >... simultaneously 30 Robustness Testing    An extension of Boundary Value testing: Include values below the minimum value and above the maximum value This is a form of what we call ―negative testing how does the program handle errors in the input? This type of testing may not be possible with some strongly-typed languages, GUIs with fixed palette values or drop-down lists, etc 31 Robustness Testing a b x Test... invalid inputs? 3 Leads to exploratory testing (test hackers) 32 Robustness Testing (2 Variables) x2 d c x1 a b 33 Worst-Case Testing Discard the Single Fault Assumption  Allow the input values to simultaneously approach their boundaries  34 Worst Case Testing (2 Variables) x2 d c x1 b a Eliminate the "single fault" assumption Murphy's Law 35 Robust Worst-Case Testing Discard the Single Fault Assumption... the minimum, at a nominal value, just below the maximum, and at the maximum 28 Input Boundary Value Testing a b x x(min) x(min+) x(nom) x(max -) x(max) Test cases for a variable x, where a ≤ x ≤ b Experience shows that errors occur more frequently for extreme values of a variable 29 Input Boundary Value Testing (2 Variables) x2 d c x1 a Test cases for variables x1 and x2, where a ≤ x1 ≤ b and c ≤ x2... sales up to and including $1000; 15% on the next $800; and 20% on any amount over $1800 This program has calculation logic to test – inputs and outputs are straightforward 23 Boundary Value testing 24 Boundary/Limits Testing    Test values, sizes or quantities near the design limits  Value limits  Length limits  Volume limits  Null strings vs Empty strings Errors tend to occur near the extreme... design 3 Form test specifications   If the specification is large, break it into independently testable features to be considered in testing Typically: combinations of input values, or model behaviors 4 Produce and execute actual tests 13 From specification to test cases Functional Specifications Independently Testable Feature Representative Values Model Test Case Specifications Test Cases 14 Simple . Functional testing 2 Contents  Introduction to functional testing  Functional testing techniques  Boundary Value testing  Equivalence Class testing  Special Value testing  Decision. testing  Decision Tables 3 Introduction to functional testing 4 Functional testing  Functional testing: Deriving test cases from program specifications  Functional refers to the source of information. the spec 11 Functional versus Structural: Classes of faults  Different testing strategies (functional, structural) are most effective for different classes of faults  Functional testing is