Dynamic techniques doc

5/16/2013 1 1 Overview 2 Life cycle 5 Management 3 Static testing 6 Tools 7 Web testing 4 Dynamic test techniques 8 Software quality Dynamic techniques Test condition – Test case – Test procedure Black-box techniques White-box techniques Experience-based techniques Choosing test techniques 1 2 3 5 6 7 4 8 Why dynamic test techniques?  Exhaustive testing (use of all possible inputs and conditions) is impractical  must use a subset of all possible test cases  must have high probability of detecting faults  Need thought processes that help to select test cases more intelligently  test case design techniques are such thought processes Slide 3 What is a testing technique?  a procedure for selecting test cases  a way of deriving good test cases  maximize possibility faults found, minimize time and resources Slide 4 5/16/2013 2 Advantages of techniques  Effective testing: find more faults  focus attention on specific types of fault  know you're testing the right thing  Efficient testing: find faults with less effort  avoid duplication  systematic techniques are measurable Slide 5 Dynamic technique Structure-based Specification-based Experience-based Statement Decision Condition Condition Combination Equivalence Partitioning Boundary Value Analysis Decision Tables Use Case testing Error guessing Exploratory testing State Transition 6 Categories of dynamic techniques  Specification-based (black-box techniques)  View software as black-box with input and output  Structure-based (white-box or glass-box techniques)  See the internal structure of the software  Experience-based  Use the tester’s experience, knowledge and intuition Where to apply? Slide 7 Where to apply?  Black box appropriate at all levels but dominates higher levels of testing  White box used predominately at lower levels  Experience-based techniques used when there is no specification or inadequate or out of date Integration Component Acceptance System Slide 8 5/16/2013 3 Dynamic techniques Test condition – Test case – Test procedure Black-box techniques White-box techniques Experience-based techniques Choosing test techniques 1 2 3 5 6 7 4 8 Analysis & Design Test process Analysis & Design Execution Evaluating Reporting Check completion Identify conditions Design test cases Specify test procedures Planning and Control Slide 10 Task 1: identify test conditions  Test condition determine 'what' is to be tested  e.g. requirements, architecture, design, interfaces  Based on  system requirement  code  experienced user's knowledge of the system  Test conditions should be able to be linked back to their sources in the test basis (traceability)  Prioritise the test conditions  must ensure most important conditions are covered Slide 11 Task 2: design test cases  A set of input values, expected results, pre-conditions, post-conditions, developed for a test condition  expected result: predict the outcome of each test case  pre-condition: specifies things that must in place before the test can be run  post-condition: specifies anything that applies after the test case completes  Prioritise the test cases Slide 12 5/16/2013 4 Task 3: specify test procedures  Describes the sequential steps that need to be done to run a set of tests  Also referred to as a test script Slide 13 Example: Check Login functionality Test Condition Test Case Name Pre- cond Test Procedure Test Data Expected Results Actual Results Status Agent Name: admin Password: 123456 1) Launch application 2) Enter Agent Name 3) Enter Password 4) Click Login Login must be successfull Check Login functionality Check valid Agent Name & Password Slide 14 Dynamic techniques Test condition – Test case – Test procedure Black-box techniques White-box techniques Experience-based techniques Choosing test techniques 1 2 3 5 6 7 4 8 Black-box techniques  Based on specifications or models of what the system should do  Known as specification-based techniques  Some techniques  Equivalence partitioning  Boundary value analysis  Decision tables testing  State transition testing  Use case testing Slide 16 5/16/2013 5 Equivalence partitioning (EP)  Divide (partition) the inputs, outputs, into areas which are the same (equivalent)  In an area, if one element works correctly, all will work correctly  One from each partition better than all from one 1 100 101 0 valid invalid invalid Slide 17 Example EP From 18 to 50 18 50 valid invalid invalid 0-9 Non numeric ==> Test cases (EP): 10, 20, 60, ab Slide 18 Age Design test case  For positive tests and for tests that you are reasonably confident will pass, the more test conditions that can be covered in a single test case, the fewer test cases will be needed  Try to cover only one invalid test condition per test case  Example Condition Test case name Inputs Expected results Test on Age field Test valid Age 20 ok Test invalid Age value 10 error message ‘Invalid value’ on Age Test invalid Age value 60 error message ‘Invalid value’ on Age Test invalid characters ab error message ‘Invalid character’ on Age Slide 19 Boundary value analysis (BVA)  Based on testing at the boundaries between partitions  Faults tend to lurk near boundaries  Good place to look for faults  Test values on both sides of boundaries 1 100 101 0 valid invalid invalid Slide 20 5/16/2013 6 Example BVA From 3 to 50 characters 3 50 valid invalid invalid ==> Test cases (BVA): 2, 3, 50, 51 2 51 Number of characters Slide 21 Customer Name Account number Loan amount requested Term of loan Repayment: Interest rate: Total paid back: 6 digits, 1 st non-zero £500 to £9000 1 to 30 years 2-64 chars Example 1: Loan application Slide 22 Customer name number of characters: valid characters 2-64 chars Slide 23 Design test case for customer name Condition Input Expected result Actual result Test valid number of chars ( 2 to 64 chars ) Customer name=John Account numer=… … Input OK Slide 24 5/16/2013 7 Account number first character: Conditions Valid Partitions Invalid Partitions Valid Boundaries Invalid Boundaries Account number 6 digits, 1 st non-zero number of digits: valid characters: Slide 25 Loan amount Conditions Valid Partitions Invalid Partitions Valid Boundaries Invalid Boundaries Loan amount £500 to £9000 Slide 26 loan amount Condition template Conditions Valid Partitions Invalid Partitions Valid Boundaries Invalid Boundaries Customer name 2 - 64 chars < 2 chars 2 chars 1 char valid chars > 64 chars 64 chars 65 chars invalid char 0 chars Account number 6 digits < 6 digits 100000 5 digits 1 st non- zero > 6 digits 999999 7 digits 1 st digit = 0 0 digits non -digit Loan amount 500 - 9000 < 500 500 499 >9000 9000 9001 non -digit null Slide 27 Example 2: Bank account  Suppose you have a bank account that the rate of interest depending on the balance in the account: a balance in the range $0 up to $100 has a 3%, a balance over $100 and up to $1000 has a 5%, and balances of $1000 and over have a 7%. What valid and invalid input partitions might you use? What test cases we design? Slide 28 5/16/2013 8 Solution: bank account Test conditions Valid partitions Invalid partitions Valid boundaries Invalid boundaries Balance in account Slide 29 Why do both EP and BVA?  If you do boundaries only, you have covered all the partitions as well  technically correct and may be OK if everything works correctly!  if the test fails, is the whole partition wrong, or is a boundary in the wrong place - have to test mid-partition anyway  testing only extremes may not give confidence for typical use scenarios (especially for users)  boundaries may be harder (more costly) to set up Slide 30 Extending EP and BVA  subjectiveness  taking on risk (need to be careful to choose classes)  it's science but it's also art  applied to other things, not only to ranges of numbers  booking a flight, with a choice of Economy/Coach, Premium Economy, Business or First Class tickets Slide 31 EP and BVA exercise  Exercise 1: Suppose you have a bank account that offers variable interest rates: 0.5% for the first £1,000.000 credit; 1% for the next £1,000.000; 1.5% for the rest. If you wanted to check that the bank was handling your account correctly what valid input partitions might you use? Slide 32 5/16/2013 9 Solution Slide 33 EP and BVA exercise  Exercise 2: A mail-order company selling flower seeds charges £3.95 for postage and packing on all orders up to £20.00 value and £4.95 for orders above £20.00 value and up to £40.00 value. For orders above £40.00 value there is no charge for postage and packing. If you were using equivalence partitioning to prepare test cases for the postage and packing charges what valid partitions would you define? What about non-valid partitions? What boundary values? Design test cases. Slide 34 Solution Test conditions Valid partitions Invalid partitions Valid boundaries Invalid boundaries Orders Slide 35 Solution - Design test cases Condition # Inputs Expected results Orders 1 £10.00 Postage and packing = 3.95 2 £30.00 Postage and packing = 4.95 3 £50.00 Postage and packing = 0 4 - £1.00 Invalid orders 5 a1 Invalid character 6 £0.00 Postage and packing = 0 7 £0.01 Postage and packing = 3.95 8 £ 20.00 Postage and packing = 3.95 9 £20.01 Postage and packing = 4.95 10 £40.00 Postage and packing = 4.95 11 £40.01 Postage and packing = 0 12 - 0.01 Invalid orders Slide 36 5/16/2013 10 Decision tables testing  A good way to deal with combination of inputs, which produce different results  Decision table  known as a 'cause-effect' table  a table showing combinations of input with their associated output or action Business Rule 1 Business Rule 2 Business Rule 3 Business Rule 4 Condition 1 T T F F Condition 2 T F T T Condition 3 T - F T Action 1 Y N N N Action 2 N Y Y N Slide 37 Decision tables testing  Design and using decision table  identify conditions (which need to be combined)  put them into a table  identify all of the combinations of true and false  identify the correct outcome for each combination  rationalise input combinations  some combinations may be impossible or not of interest  some combinations may be ‘equivalent’  use a hyphen to denote “don’t care”  write test cases for each of the rule in the table  each column is at least one test case Slide 38 Rationalising outputs  If outputs or effects are mutually exclusive, i.e. T occurs in only one place in each column, we can combine them  For example: Y F T F X T F F Z F F T is equivalent to: Output X Y Z Slide 39 Decision tables testing example 1 Take an example of a loan application, where you can enter the number of years you want to take to pay it back (the term of the loan) or the amount of the monthly repayment. You are not allowed to enter both repayment and term. Term of loan Monthly repayment Slide 40 [...]... very complex Slide 68 17 5/16/2013 1 2 3 4 5 6 7 8 White-box techniques  Structure-based approach  based on the internal structure of a component or system  also called glass-box techniques Dynamic techniques Test condition – Test case – Test procedure Black-box techniques White-box techniques Experience-based techniques Choosing test techniques  What we may be interested in structures  component... Slide 100 25 5/16/2013 1 2 3 4 5 6 7 8 Non-systematic test techniques  Based on a person's knowledge, experience, imagination and intuition  Some techniques  error guessing  exploratory testing Dynamic techniques Test condition – Test case – Test procedure Black-box techniques White-box techniques Experience-based techniques Choosing test techniques It is true that testing should be rigorous, thorough... test techniques that explorers can use  combines with formal testing techniques  Most useful when there are no or poor specifications and when time is severely limited Slide 105 1 2 3 4 5 6 7 Slide 106 8 Choosing test techniques  Each technique is good for certain things, and not as good for other things Dynamic techniques Test condition – Test case – Test procedure Black-box techniques White-box techniques. .. techniques Experience-based techniques Choosing test techniques  The best testing technique is no single testing technique  Each testing technique is good at finding one specific class of defect  How can we choose the most appropriate testing techniques to use?  Depend on internal and external factors Slide 108 27 5/16/2013 Choosing test techniques (cont’d) Choosing test techniques (cont’d)  Internal... division by zero  past failures  blank (or no) input  intuition  empty files  wrong kind of data (e.g alphabetic characters where numeric are required)  After systematic techniques have been used  Supplements systematic techniques Not a good approach to start testing with Slide 103 Slide 104 26 5/16/2013 Exploratory testing Exploratory testing  Exploratory testing as 'an interactive process... structures  component level: program structures  integration level: the way components interact with others  system level: how user will interact with the system Slide 70 White-box techniques Statement testing  Some techniques  control flow testing  Design test cases based on what statements which       should be exercised  Statement coverage  the percentage of executable statements exercised... not valid S: Displays message and rejects card 4a PIN not valid S: Displays message and ask for re-try (twice) PIN invalid 3 times Slide 66 S: Eats card and exit 4b Slide 65 Black-box techniques - Advantages Black-box techniques - Disadvantages  More effective on larger units of code than glass box  Only a small number of possible inputs can actually be      testing Tester needs no knowledge... techniques to use?  Depend on internal and external factors Slide 108 27 5/16/2013 Choosing test techniques (cont’d) Choosing test techniques (cont’d)  Internal factors  models used  test objective  documentation  life cycle model  tester knowledge or experience  External factors  risk  customer or contractual requirements  type of system  regulatory requirements  time and budget Slide 109 . testing 4 Dynamic test techniques 8 Software quality Dynamic techniques Test condition – Test case – Test procedure Black-box techniques White-box techniques Experience-based techniques. Password Slide 14 Dynamic techniques Test condition – Test case – Test procedure Black-box techniques White-box techniques Experience-based techniques Choosing test techniques 1 2 3 5. 68 5/16/2013 18 Dynamic techniques Test condition – Test case – Test procedure Black-box techniques White-box techniques Experience-based techniques Choosing test techniques 1 2 3 5