The IEEE 829 standard can serve as a guideline while using automated systems for bug reporting and tracking. As seen in figure 11.9, the information related to bug tracking is filled in the form which is primarily text and numbers. This information can easily be recorded on a database. The main window of a typical bug-reporting database shows what an automated system can provide.
Figure 11.8 an automated bug reporting and tracking system.
Figure 11.8 A Sample Bug Reporting Database
Source:
Figure 11.8 illustrates a bug reporting and tracking database. A glance at the screen is enough to know the status of bugs. The bug’s description includes its type, component, priority, environment, target version, fix version, and so on.
http://www.bug-track.com/dashboard/viewDashboard.do?action=viewOpenBugsDashboard
Figure 11.9 shows the New Bug Dialog Box, in which information is entered to record a new bug into the system.
Figure 11.9 A Sample New Bug Dialog Box
Source:
This dialog box is typically used by the programmer to record information concerning the bug fix. A drop-down list supplies different types of bug and their status. The bug is then given back to the tester for closing.
http://www.bug-track.com/bugs/editBugs.do?action=Create
Figure 11.10 shows the dialog box used when a bug is resolved by a programmer or project manager.
Figure 11.10 A Sample Dialog Box used when a Bug is Resolved by a Programmer or Project Manager
Source:
resolution&user_query_key=main&FromNavURL=&FromNav=&itemsOffSet=&scrollLoc=0# http://www.bug-track.com/bugs/editBugs.do?action=Edit&key=148950&drill=&drill_key=null&Section=
Several bug reporting and tracking databases track not just comments about the fix, but also details of what exactly the programmers did to make the fix. Herein, the line number, the module, and even the type of error can be recorded.
After a bug is resolved, it is given back to the tester for closing. Hence one will have a bug report containing the entire test history for review.
As the database has tracked every alteration to the bug report since it was opened, it helps to view the decisions that were made along the way and review what was fixed. It is possible that the bug was not fixed as expected, possibly a similar bug had been found and added by another tester, or maybe the programmer made a comment about the fix being risky. All this information will assist when a retest of the bug is done to make sure it is fixed. If it is found that it is not fixed, the bug is reopened to start the life cycle all over again.
Figure 11.11 shows the bug closing dialog box.
Figure 11.11 A sample Bug Report, Ready for closing
Source:
A bug-tracking database helps to organize the entire project team. By using a bug-tracking database, testers can communicate the status of the project, know who is assigned what task, and most importantly, assure that no bug slips occur.
Mantis Bug Database Images Courtesy of Dave Ball and HBS International, Inc.
Microsoft Access and Bugzilla are very popular bug tracker databases. FogBugz is another example.
• Test planning is the basic test documentation that contains the record of the testing effort.
11.3 Summary
• The goal of a test plan is to facilitate communication between the tester and programmer. Both cannot work in isolation.
• The IEEE Standard 829 for Software Test Documentation states that the purpose of a software test plan is to prescribe the scope, approach, resources, and schedule of the testing activities.
• To plan the test stages, the test team analyzes the selected development model and decides whether certain stages of testing should be performed during the course of the project.
• The testing schedule outlines the duration of each test that will be performed on the module.
• The four features that make it necessary to plan the test cases are its organization, repeatability, tracking, and proof of testing.
• Test design specifications are used to organize and describe the testing that needs to be performed on a specific module.
• The main aim of a bug report is to let the programmer know that some part of the software is not giving the expected result.
• Bug tracking and reporting can be carried out manually or by using automated tools.
11.4 Keywords
Database: A database is a collection of data and a system intended to organize and retrieve huge
amounts of data quickly and easily.
Log: It is a detailed record of events and actions.
Spreadsheet: Spreadsheets, also known as worksheets, have rows and columns that make it easy to
display information to insert formulas and work with the data.
Testopia: Testopia is a test case management extension for Bugzilla, which is designed to be a generic
tool for tracking test cases. It allows testing organizations to integrate bug reporting with their test case run results.
IEEE 829: IEEE 829-1998, also known as the 829 Standard for Software Test Documentation, is an IEEE
standard. This specifies the form of a set of documents for use in eight defined stages of software testing. Each stage produces its own separate type of document.
1. State whether the following statements are true or false:
11.5 Self Assessment
(a) IEEE 829 states that the test procedure specification identifies all the steps required to operate the system. It describes how the tester will physically run the test, the physical set- up required, and the procedure steps that need to be followed.
(b) A bug retains the same form throughout its life cycle.
(c) If the tester thinks the bug is legitimate, then the state of the bug is changed to
“REJECTED”.
(d) The IEEE 829 standard does not define the format that the bug report should follow.
2. Fill in the blanks:
(a) A detailed __________ will allow a tester to understand exactly what will be tested and how it will be tested.
(b) A bug’s state will be ______ when it is posted for the first time.
(c) When the bug is accepted by the project manager, its state is _____________.
3. Select the suitable choice for every question:
(a) Planning test cases systematically is important for _________many reasons.
(i) Two (ii) Three (iii) Four (iv) Five
(b) IEEE 829-1998 Standard for Software Test Documentation is used widely by many testing teams because:
(i) It is logical and reasonable (ii) It is easy to follow (iii) It is incorporated in a tool (iv) None of the above
(c) The test procedures document should specify a list of (i) Procedures and Environmental needs
(ii) Special requirements and Procedure (iii) Identifiers and Input specifications (iv) Test items and Approach
(d) The state of the bug is set to “Deferred” when:
(i) The developer fixes the bug (ii) The bug is still not fixed (iii) The bug is repeated twice (iv) The bug is decided to be fixed in the next release
1. “Precise and systematic planning of test cases is a step in making the testing process disciplined.”
Discuss the reasons for test case planning.
11.6 Review Questions
2. “IEEE 829 states that the test design specification improves the test approach (defined in the test plan) and finds the modules to be covered by the design and its associated tests.” Elaborate.
3. “A bug-tracking database organizes the entire project team.” Explain briefly how a bug tracking system helps.
4. IEEE 829 states that “the test procedure specification identifies all the steps required to operate the system. It describes how the tester will physically run the test, the physical set-up required, and the procedure steps that need to be followed.” Explain.
5. “It is not enough for a test procedure to ask a tester to try all the test cases and report the observations. “ Justify.
6. Do you think all bugs can be fixed? If not, list some reasons why some bugs might not be fixed.
7. “A preferred and practical method of tracking test cases is by using a spreadsheet.” Discuss.
8. “One consideration that must be taken into account when creating the test case documentation is how the information will be organized and tracked.” Do you agree? Justify.
9. “IEEE 829 standard is widely used by many testing teams” Explain why?
10. “If testers want the software development process to be disciplined, they use the four methods”.
Discuss.
11. “In some cases, the life cycle of a bug gets a bit more complicated.” Do you agree? Justify.
12. “The IEEE 829 Standard for Software Test Documentation defines a document called the Test Incident Report.” Briefly discuss the purpose of the Test Incident Report.
Answers: Self Assessment
1. (a) True (b) False (c) False (d) True 2. (a) Planning (b) New (c) Open
3. (a) Four (b) It is logical and reasonable (c) Special requirements and Procedures (d) The bug is decided to be fixed in the next release
11.7 Further Readings
Daniel Galin, Software Quality Assurance-From theory to implementation, Pearson Ron Patton, Software Testing, Second Edition, Sams Publishing
http://www.exforsys.com/tutorials/testing/bug-life-cycle-guidelines.html http://www.scribd.com/doc/16103218/Software-Testing-Manual
http://www.nickjenkins.net/prose/testingPrimer.pdf
CONTENTS
Objectives Introduction 12.1 Definition of Quality 12.2 Testing and Quality Assurance at Workplace 12.2.1 Difference between Software Testing and Quality Assurance 12.3 Quality Management in IT
12.4 Summary 12.5 Keywords 12.6 Self Assessment 12.7 Review Questions 12.8 Further Readings
Unit 12: Software Quality Assurance
After studying this unit, you will be able to:
Objectives
• Define quality
• Explain testing and quality assurance at workplace
• Explain quality management in IT
Quality is defined as the features or the attributes of the products that are appreciated by the end-users or the customers. “Quality means conformance to requirements “as indicated by Crosby in 1979.
Quality assurance is an orderly procedure of inspecting a particular product or a service that is being developed to meet the required standards. Many organizations allocate a whole unit for quality assurance purposes. A good quality assurance system not just enhances the organization’s credibility, it also builds customer’s belief, thereby improving the process which helps the organization to compete with others.
Introduction
Did you know? During World War II, military weapons were checked and tested for defects after they
were developed. However, in today’s scenario quality assurance systems emphasize on identifying the defects before the development of the final product.
Quality assurance is considered to be the most important activity for any business involved in software development. The history of quality assurance in software development is similar to that of the history of quality in hardware manufacturing. Software quality assurance is defined as a planned and methodical pattern of actions used to ensure the quality of the product as per the standards established.
In order to follow quality guidelines, a company's management team frames quality assurance policies and objectives. The company’s external consultant or management writes down the company policies and requirements in a structured format, as to how the staff can implement the quality assurance system. Once this guideline is framed and quality assurance procedures are implemented, an external evaluator examines the company's quality assurance system to ensure its conformance with the set standards such as ISO or CMM.
In the year 2009 – 2010, Toyota Motors had recalled millions of its vehicles.
This is because most of the vehicles had experienced an increased acceleration.
This was due to incorrect placement of the driver’s front floor mat at the foot pedal well. The wrong placement resulted in pedal entrapment when the vehicle was in motion. However, on investigation, it was found that the quality of the foot mat was low and this had caused the unintended acceleration in the vehicle.
“Quality is the enduring process of building and sustaining relationships by assessing, anticipating, and fulfilling stated and implied needs”Winder, Richard E. and Judd, Daniel K., 1996.
Quality can be defined as a state of being free from defects and deficiencies. It is achieved by adopting strict and consistent adherence to measure and verify the set standards to attain uniformity in the output which satisfies specific user requirements.
ISO defines quality as "the totality of features and characteristics of a product or service that bears its ability to satisfy stated or implied needs”. It is the state of being free from defects and deficiencies by adopting strict and consistent adherence to measure and verify the set standards in order to attain uniformity in the output. This in turn satisfies specific user requirements.
To understand the meaning and importance of software quality in the software industry, we first need to understand the activities related to quality such as software quality assurance and software quality control.