Lecture Software process improvement: Lesson 40B provide students with knowledge about: a goal-based framework for software measurement; classifying software measures; internal attributes; average cost of each defect detected during the process;... Please refer to the detailed content of the lecture!
A Goalbased Framework for Software Measurement Lecture # 40B A Goalbased Framework for Software Measurement • Today, we’ll talk about a conceptual framework for the diverse software measurement activities that contribute to an organization’s software practices • These practices may include not only your usual development and maintenance activities, but also any experiments and case studies you may perform as you investigate new techniques and tools A Goalbased Framework for Software Measurement • This framework is based on three principles: – Classifying the entities to be examined – Determining relevant measurement tools – Identifying the level of maturity that your organization has reached Classifying Software Measures • The first obligation of any software measurement activity is identifying the entities and attributes we wish to measure • In software, there are three such classes (discussed next): • Processes – These are collections of softwarerelated activities • Products – These are any artifacts, deliverables or documents that result from a process activity • Resources – These are entities required by a process activity • A process is usually associated with some timescale, i.e., the activities in the process are ordered or related in some way that depends on time, so that one activity must be completed before another can begin • The timing can be explicit, as when design must be complete by October 31, or implicit, as when a flow diagram shows that design must be completed before coding can begin • Resources and products are associated with the process • Each process activity has resources and products that it uses, as well as products that are produced • Thus, the product of one activity may feed another activity, e.g., the design document • Before, we discuss these entities in detail, let’s distinguish between internal and external attributes: – Internal attributes – External attributes Internal Attributes • Internal attributes of a product, process or resource are those that can be measured purely in terms of the product, process or resource itself • In other words, an internal attribute can be measured by examining the product, process or resource on its own, separate from its behavior 10 • Finally, you analyze each question in terms of what measurements you need in order to answer each question • Once you have a list of possible measures, you must assess your organization to see whether it is capable of providing useful information for the measurement 63 • For example, if your developers cannot tell when design ends and coding begins, then measuring the duration of the design activity will be impossible • To understand when measurement is appropriate for your organization, you must know the process maturity of your organization • The more mature your process, the more 64 that is visible and therefore measurable GQM Paradigm • A measurement program can be more successful if it is designed with the goals of the project in mind • The GQM approach provides a framework involving three steps (shown on next slide) 65 Steps of GQM Framework 66 Steps of GQM Framework • List the major goals of the development or maintenance project • Derive from each goal the questions that must be answered to determine if the goals are being met • Decide what must be measured in order to be able to answer the questions adequately 67 • By deriving the measurements in this way, it becomes clear how to use the resulting data • Several metrics might be generated from a single goal • One metric may be supporting several goals 68 • Suppose your overall goal is to evaluate the effectiveness of using a coding standard • First, it is important to know who is using the standard, so that you can compare the productivity of those who do not. You would also want to compare the quality of the code produced with the standard with the quality of nonstandard code 69 • Second, you must analyze each question to determine what must be measured in order to answer the question • For example, to understand who is using the standard, it is necessary to know what proportion of coders is using the standard. It is also important to have an experience profile of coders, explaining how long they have worked with the standard, the environment, the language, and other factors that will help 70 evaluate the effectiveness of the standard • The productivity question requires a definition of productivity, which is usually some measure of effort divided by some measure of product size (in LOC, FPs, …) • Quality may be measured in terms of the number of errors found in the code, plus any other quality measures that you would like to use 71 • In this way, you generate only those measures that are related to the goal • In many cases, several measurements may be needed to answer a single question • Likewise, a single measurement may apply to more than one question • The goal provides the purpose for collecting the data, and the questions tell you and your 72 project how to use the data Deriving Metrics from Goals and Questions 73 Deriving Metrics from Goals and Questions GOAL: Evaluate effectiveness of coding standard QUESTIONS: Who is using standard? METRICS: Proportion of coders using standard using language What is coder productivity? What is code quality? Code Effort Experience Size (LOC, of coders FPs, etc.) with standard with language with environment, etc Errors… 74 Discussion on this Example 75 Summary 76 References • Software Metrics: A Rigorous & Practical Approach, by Norman E. Fenton and Shari L. Pfleeger, 2nd Edition, PWS Publishing Company, 1997(Chapter 3.13.2 – up to page 85) 77 ... expended per thousand lines of code reviewed 21 Effectiveness of? ?Software? ?Testing Process 22 Effectiveness of? ?Software? ? Testing? ?Process The testing? ?process? ?may be composed of unit testing, integration testing, system testing, and acceptance ... directly the attributes of interest 13 Processes (3.1.1) • We often have questions about our? ?software? ? development activities and processes that measurement can help us answer • We want to know how long it takes for a process? ?to complete, how much it will cost, ... control the? ?process? ?by telling us how the process? ?is using and changing inputs to outputs 49 • For example, if we are producing poor quality? ?software, resource measurements may show us that the? ?software? ?quality is the