278 Chapter 19 ■ Testing made considerably easier. Various approaches are explained in Chapter 24 on incre- mental development. We have seen that exhaustive testing is infeasible. Therefore complete testing is impossible and, whatever testing methods are used, they can never ensure that the software is free from bugs. Thus testing is a poor technique but until formal verification becomes wide- ly applicable it is a vital technique. However much we test our programs, using all our skill and intuition, we can never be sure that we have eradicated all the faults. The situation is well summed up by one of computing’s gurus, Dijkstra, in his famous remark, “Testing can only show the pres- ence of bugs, never their absence.” This has been (anonymously) rephrased as, “Just because you have never seen a mermaid doesn’t mean that they don’t exist.” It can be reassuring to adopt the view that a test that reveals no bugs is a successful test. But rather we should look upon such a test as unsuccessful! It is difficult to get accurate data on the number of bugs present in production soft- ware because, unsurprisingly, organizations do not want to reveal this kind of informa- tion. The indications are that there are typically between 2 and 50 bugs per 1,000 lines of source code in commercial production software. A figure like this is more properly called a fault density. It measures the number of known faults per 1,000 lines of code (LOC). A figure of 2 is considered to be most creditable. Ways of measuring this quan- tity are explained in Chapter 29 on metrics and quality assurance. The trouble is, of course, that bugs always surface at the worst possible time, for example, when you are demonstrating the completed software to the client. This phe- nomenon has long been known to students of reliability, who quote Murphy’s laws: 1. “If a system can fail, it will,” 2. “and at the worst possible moment.” Another, more objective, observation is that some bugs create serious faults, while others lie dormant and do not give any trouble. Chapter 31 on assessing methods looks at the evidence that is available to compare verification techniques, including testing. The surprising indications are that simply inspecting code is more effective than carrying out testing. The worrying conclusion to any discussion of verification is that all software (of any significant size) contains faults. 19.9 ● Discussion Summary Testing is one set of techniques for verifying software. Exhaustive testing is a practical impossibility. BELL_C19.QXD 1/30/05 4:25 PM Page 278 Exercises 279 19.1 Consider a program that has 16 if-then statements in it. Then there are 2 16 possi- ble paths through it. If each test takes 50 microseconds and each action takes 50 microseconds (a gross underestimate), how much computer time is needed to test all program paths? 19.2 Devise black box and white box test data to test the following program. The program specification is: The program inputs a series of integers from the keyboard using a text field. The program finds the largest of the numbers. The numbers are terminated when a but- ton labeled Start Again is pressed. Try not to look at the text of the program, given below, until you have completed the design of the black box data. The program involves the following class: class Biggest { private int largest; public Biggest() { largest = 0; } public void nextNumber(int n) { if (n > largest) largest = n; } public void display(TextField textField) { textField.setText("largest so far is" + largest); } public void startAgain() { largest = 0; } } Exercises In black box (or functional) testing, sample data based on the specification is used. This is termed equivalence partitioning. In white box (or structural) testing, the internal structure of the software is used to select test data. This means that every path through the program is tested. Unit testing tests each component in isolation. Drivers and stubs are used to sub- stitute for missing components. Integration testing tests components as they are brought together. • BELL_C19.QXD 1/30/05 4:25 PM Page 279 280 Chapter 19 ■ Testing 19.3 Devise black box and white box test data to test the following program. The program specification is: The program is to determine insurance premiums for a holiday, based upon the age and gender (male or female) of the client. For a female of age >= 18 and <= 30 the premium is $5. A female aged >= 31 pays $3.50. A male of age >= 18 and <= 35 pays $6. A male aged >= 36 pays $5.50. People aged 50 or more pay half premium. Any other ages or genders are an error, which is signaled as a premium of zero. The Java code for this program is: public float calcPremium(float age, String gender) { float premium; if (gender.equals("female")) if ((age >= 18) && (age <= 30)) premium = 5.0f; else if (age >= 31) premium = 3.50f; else premium = 0.0f; else if (gender.equals("male")) if ((age >= 18) && (age <= 35)) premium = 6.0f; else if (age >= 36) premium = 5.5f; else premium = 0.0f; else premium = 0.0f; if (age >= 50) premium = premium * 0.5f; return premium; } 19.4 Suggest features for software tools that could assist in using each of the following techniques: ■ black box testing ■ white box testing. 19.5 Substantial testing of a system uncovers not a single error. What conclusions would you draw? BELL_C19.QXD 1/30/05 4:25 PM Page 280 Answers to self-test questions 281 Answers to self-test questions 19.1 A row number is in three partitions: 1. within the range 1–8 2. less than 1 3. greater than 8. If we choose one representative value in each partition (say 3, –3 and 11 respectively) and a similar set of values for the column numbers (say 5, –2 and 34), the test data will be: 135OK 2 –3 5 invalid 3 11 5 invalid 4 3 –2 invalid 5 –3 –2 invalid 6 11 –2 invalid 7 3 34 invalid 8 –3 34 invalid 9 11 34 invalid Test number Row Column Outcome We now remember that data near the boundary of the partitions is important and therefore add to the test data for each partition so that it becomes: 1. within the range 1–8 (say 3) 2. less than 1 (say –3) 3. greater than 8 (say 11) 4. boundary value 1 5. boundary value 8 6. boundary value 0 7. boundary value 9. which now gives many more combinations to use as test data. ➞ BELL_C19.QXD 1/30/05 4:25 PM Page 281 282 Chapter 19 ■ Testing This book surveys studies of the types of fault that occur and explains the different test- ing methods, in a very readable way: Marc Roper, Software Testing, McGraw-Hill, 1994. A readable practical review of testing techniques: Cem Kaner, Testing Computer Software, John Wiley, 1999. The following book describes lessons in debugging and testing learned at Microsoft. The author, Steve Maguire, is a strong advocate of stepping through code using the debugger as a good way of finding bugs. The examples given are in C: Steve Maguire, Writing Solid Code, Microsoft Press, 1993. 19.2 There are four paths through the program, which can be exercised by the following test data: 13213 23255 32313 42355 Test number Outcome 19.3 There are three paths through the program extract, including the path where neither of the conditions in the if statements are true. But each of the error messages can be triggered by two conditions. Suitable test data is therefore: 156OK 2 0 4 invalid 3 9 4 invalid 4 5 9 invalid 5 5 0 invalid Test number Row Column Outcome Further Reading • BELL_C19.QXD 1/30/05 4:25 PM Page 282 This chapter is about collaborative ways of working – both informal and semi-formal. We look at structured walkthroughs, inspections and pair programming. These aim to improve software productivity and quality, and perhaps also enhance the enjoyment of programming. Programmers are often seen as loners. Given a clear specification, a programmer often carries out the complete process of program design, coding and testing entirely on their own. Programmers are seen as low-profile technicians in contrast to the articulate extro- vert systems analysts. Thus a program is sometimes seen as a personal work of art, the creation of an individual programmer. These attitudes deny that “two heads are better than one”, that through discussion with others we can produce better work. The common experience that someone else can spot errors better than the author lead to the invention of the structured walkthrough. Credit for its invention belongs to G. Weinberg, in his book The Psychology of Computer Programming. Weinberg suggest- ed that programmers see their programs as an extension of themselves. He suggested that we get very involved with our own creations and tend to regard them as manifesta- tions of our own thoughts. We are unable to see mistakes in our own programs, since to do so would be to find a fault in ourselves, and this, apparently, is unacceptable to us. 20.2 ● The individual and the error 20.1 ● Introduction CHAPTER 20 Groups This chapter explains: ■ how to use structured walkthroughs ■ how to use inspections ■ how to carry out pair programming. BELL_C20.QXD 1/30/05 4:25 PM Page 283 284 Chapter 20 ■ Groups The term for this is cognitive dissonance. The solution is to seek help with fault finding. In doing this we relinquish our private relationship with our work. Programming becomes ego-less programming. This is a completely informal technique, carried out by colleagues in a friendly manner. It is not a formalized method carried out at fixed times and made into a rigid procedure of the organization. Indeed, to formalize ego-less programming would be to destroy its ethos and therefore its effectiveness. If you get a friend or a colleague to inspect your program, it is extraordinary to witness how quickly someone else can see a fault that has been defeating you for hours. Studies also show that different people tend to uncover different types of fault. This further suggests the use of team techniques. This is simply the term for an organized meeting at which a program (or some other product) is examined by a group of colleagues. The major aim of the meeting is to try to find bugs which might otherwise go undetected for some time. (There are other goals, which are explained later.) The word “structured” simply means “well organ- ized”. The term “walkthrough” means the activity of the programmer explaining step by step the working of his/her program. The reasoning behind structured walk- throughs is just this: that by letting other people look at your program, errors will be found much more quickly. To walkthrough a program you need only: ■ the specification ■ the text of the program on paper. In carrying out a walkthrough, a good approach is to study it one method at a time. Some of the checks are fairly straightforward: ■ variables initialized ■ loops correctly initialized and terminated ■ method calls have the correct parameters. Another check depends on the logic of the method. Pretend to execute the method as if you were a computer, avoiding following any calls into other methods. Check that: ■ the logic of the method achieves its desired purpose. During inspection you can also check that: ■ variable and method names are meaningful ■ indentation is clear and consistent. The prime goal of a walkthrough is to find bugs, but checking for a weakness in style may point to a bug. The evidence from controlled experiments suggests that walkthroughs are a very effective way of finding errors. In fact walkthroughs are at least as good a way of iden- tifying bugs as actually running the program (doing testing). 20.3 ● Structured walkthroughs BELL_C20.QXD 1/30/05 4:25 PM Page 284 20.3 Structured walkthroughs 285 Although structured walkthroughs were initially used to find bugs in program code, the technique is valuable for reviewing the products at every stage of development – the requirements specification, a software specification, architectural design, component design, the code, the test data, the results of testing, the documentation. There are several key points in organizing walkthroughs successfully: ■ gauge the size and membership of the group carefully so that there are plenty of ideas, but so that everyone is fully involved ■ expect participants to study the material prior to the meeting ■ concentrate attention on the product rather than the person, to avoid criticizing the author ■ limit the length of the meeting, so that everyone knows that it is business-like ■ control the meeting with the aid of agreed rules and an assertive chairperson ■ restrict the activity to identifying problems, not solving them ■ briefly document the faults (not the cures) for later reference. The benefits of structured walkthroughs can be: 1. software quality is improved because ■ more bugs are eliminated ■ the software is easier to maintain, because it is clearer 2. programmer effort is reduced because ■ specifications are clarified before implementation ■ errors are detected early, and so costly rework is avoided ■ the time spent at the meeting (and in preparation for it) is more than repaid in time saved 3. meeting deadlines is improved because ■ visibility of the project is better (so potential catastrophes are prevented) ■ major errors are avoided early 4. programmer expertise is enhanced because ■ everyone learns from everyone else 5. programmer morale is improved because ■ people gain satisfaction from better work ■ people get to find out what is going on ■ people enjoy the discussions with colleagues. BELL_C20.QXD 1/30/05 4:25 PM Page 285 286 Chapter 20 ■ Groups Of course walkthroughs do mean that the individual has to be relaxed about pre- senting their work to colleagues. These are similar to structured walkthroughs – a group of people meet to review a piece of work. But they are different from walkthroughs in several respects. Checklists are used to ensure that no relevant considerations are ignored. Errors that are discovered are clas- sified according to type and carefully recorded on forms. Statistics on errors are com- puted, for example in terms of errors per 1,000 lines of code. Thus inspections are not just well organized, they are completely formal. In addition management is informed of the results of inspections, though usually they do not attend the meeting. Thus inspec- tions are potentially more threatening to the programmer than walkthroughs. There are other, minor, differences between inspections and walkthroughs. Normally there are only four members in an inspection team: ■ the moderator, who co-ordinates activities ■ the person who designed the program component being inspected ■ the programmer ■ the tester – a person who acts as someone who will be responsible for testing the component. The essence of inspections is that the study of products is carried out under close management supervision. Thus inspections are overtly a mechanism for increased con- trol over programmers’ work, similar to the way that quality control is carried out on a factory floor. Some programmers might feel threatened in this situation and become defensive, perhaps trying to hide their mistakes. Perhaps this makes the discovery of errors more painful, and programming a less enjoyable activity. From an organizational point of view, keeping records of faults discovered during inspections provides information to predict the quality of the software being written. Also, by highlighting common mistakes it can be used to improve programmers self- awareness and thereby improve their skills. Here two people sit down together, looking at the same computer screen, and carry out programming, including writing and applying tests. The people have different roles. One has the use of the keyboard and mouse. They are thinking and working on the implementation of the current piece of code (usually a particular method). They are thinking on a small scale and locally. The other person does two things: ■ observe, looking for errors ■ think more strategically. 20.5 ● Pair programming 20.4 ● Inspections BELL_C20.QXD 1/30/05 4:25 PM Page 286 Summary 287 The strategic thinking considers the role of the method within the class, and whether this method is appropriate within the context of the whole system. The person thinks about such questions as: Which other components use it? Can the system be simplified so that this method is just not needed? Can the method be generalized so that it is more wide- ly useful? Is this method in the right place, or should it be relocated into some other class? The pair periodically switch roles, so that overall they are working as equals. Pairs don’t stay together, but change from day to day. It simply depends on who is available, but the people are drawn from the same project team, so they have a shared under- standing of the project. And no one forces people to pair up if they do not get along. When a pair starts work, it may be that one person will have more experience or expertise. Later, the gap narrows, so that they are genuinely collaborating on an equal basis, one person’s strengths compensating for the other’s weaknesses. The central feature of pair programming is that two people are communicating intensely, sharing ideas, learning from each other, supporting each other, articulating ideas verbally and solving problems. It is creative, sociable, enjoyable and effective. It is claimed that pair programming improves productivity, code quality and job satisfaction. Even though, at first sight, twice the effort is spent on development, the time is more than reclaimed in more effective working. Of course, pair programming means that pairs of people are working closely – almost intimately – together. This depends on an organizational culture that is collaborative and supportive. Perhaps the ultimate method for collaborative working aimed at reducing bugs is open source development. This is such an important technique that we devoted a whole sep- arate chapter to the topic. How effective are the techniques of walkthroughs, inspections and pair program- ming? The answer is that they are surprisingly effective as compared with using testing. We review the evidence in Chapter 31 on assessing methods. 20.6 ● Discussion Summary A structured walkthrough is a meeting at which a document is examined by a group of people in order to find errors. Structured walkthroughs are based on the prem- ise that ideas that are shared will be the better for it. The careful organization of a walkthrough is important. Walkthroughs can lead to improved software quality – reliability and maintainability – because of the scrutiny of project material by a group. Effort can be reduced and deadlines more easily met. Inspections are a more formal approach to a group review meeting. In pair programming, two people sit at the computer, working closely together on design, coding and testing. BELL_C20.QXD 1/30/05 4:25 PM Page 287 . – reliability and maintainability – because of the scrutiny of project material by a group. Effort can be reduced and deadlines more easily met. Inspections are a more formal approach to a group. and therefore add to the test data for each partition so that it becomes: 1. within the range 1–8 (say 3) 2. less than 1 (say –3) 3. greater than 8 (say 11) 4. boundary value 1 5. boundary value. $5. A female aged >= 31 pays $3.50. A male of age >= 18 and <= 35 pays $6. A male aged >= 36 pays $5.50. People aged 50 or more pay half premium. Any other ages or genders are an error, which