Lecture Software process improvement: Lesson 41 provide students with knowledge about: Goal-question-metric (GQM); steps of GQM framework; deriving metrics from goals and questions; goal question metric approach; conceptual level (goal);... Please refer to the detailed content of the lecture!
GoalQuestionMetric (GQM) Lecture # 41 Steps of GQM Framework • List the major goals of the development or maintenance project • Derive from each goal the questions that must be answered to determine if the goals are being met • Decide what must be measured in order to be able to answer the questions adequately • By deriving the measurements in this way, it becomes clear how to use the resulting data • Suppose your overall goal is to evaluate the effectiveness of using a coding standard • First, it is important to know who is using the standard, so that you can compare the productivity of those who do not • You would also want to compare the quality of the code produced with the standard with the quality of nonstandard code • Second, you must analyze each question to determine what must be measured in order to answer the question • For example, to understand who is using the standard, it is necessary to know what proportion of coders is using the standard • It is also important to have an experience profile of coders, explaining how long they have worked with the standard, the environment, the language, and other factors that will help evaluate the effectiveness of the standard • The productivity question requires a definition of productivity, which is usually some measure of effort divided by some measure of product size (in LOC, FPs, …) • Quality may be measured in terms of the number of errors found in the code, plus any other quality measures that you would like to use • In this way, you generate only those measures that are related to the goal • In many cases, several measurements may be needed to answer a single question • Likewise, a single measurement may apply to more than one question • The goal provides the purpose for collecting the data, and the questions tell you and your project how to use the data Deriving Metrics from Goals and Questions Deriving Metrics from Goals and Questions GOAL: Evaluate effectiveness of coding standard QUESTIONS: Who is using standard? METRICS: Proportion of coders using standard using language What is coder productivity? What is code quality? Code Effort Experience Size (LOC, of coders FPs, etc.) with standard with language with environment, etc Errors… 10 Another Example of GQM Improve timelines of change request processing 43 GQM Goals • A GQM model is a hierarchical structure starting with a goal (specifying purpose of measurement, object to be measured, issue to be measured, and viewpoint from which the measure is taken) • So a GQM goal has three coordinates: • 1. Issue: Timeliness • 2. Object (process): Change request processing • 3. Viewpoint: Project manager – and a purpose 44 GQMGoals 45 GQM Goals Goal Improve timelines of change request processing Purpose Issue Object Improve The timelines Change request processing Viewpoint Project Manager 46 GQM Questions • The goal is refined into several questions, that usually break down the issue into its major components • We’ll try to ask at least three groups of questions 47 GQM – QuestionsGroup 1 • How can we characterize the object (product, process, or resource) with respect to the overall goal of the specific GQM model? • Example: – What is the current change request processing speed? – Is the (documented) change request process actually performed? 48 GQM – QuestionsGroup 2 • How can we characterize the attributes of the object that are relevant with respect to the issue of the specific GQM model? • Example: – What is the deviation of the actual change request processing time from the estimated one? – Is the performance of the process improving? 49 GQM – QuestionsGroup 3 • How do we evaluate the characteristics of the object that are relevant with respect to the issue of the specific GQM model? • Example: – Is the current performance satisfactory from the viewpoint of the project manager? – Is the performance visibly improving? 50 GQM Metrics • Once the questions have been developed, we proceed to associating the question with appropriate metrics. The factors we consider in doing this are many; among them: – Amount and quality of the existing data: we will try to maximize the use of existing data sources if they are available and reliable; – Maturity of the objects of measurement: we will apply objective measures to more mature measurement objects, and we will use more subjective evaluations when we deal with informal or unstable objects – Learning process: GQM models need always refinement and adaptation, therefore the measures we define must help us in evaluating not only the object of measurement but also the reliability of the model used to evaluate it 51 GQM Model 52 GQM – Model 1 Goal Purpose Improve Issue The timelines of Object Change request processing Viewpoint Project Manager Question Q1 What is the current change request processing speed? Metrics M1 Average Cycle time M2 Standard deviation M3 %age cases outside of upper limit Question Q2 Is the (documented) change request process actually performed? Metrics M4 Subjective rating by Project Manager M5 %age of exceptions identified during reviews 53 GQM – Model 2 Question Q3 What is the deviation of the actual change request processing time from the estimated? Metrics M6 ((current average cycle time – estimated average cycle time)/Current average cycle time)*100 M7 Subjective rating by Project Manager Question Q4 Is the performance of the process improving? Metrics M8 (current average cycle time/Baseline average cycle time)*100 Question Q5 Is the current performance satisfactory from the viewpoint of the project manager? Metrics M9 Subjective rating by Project Manager Question Q6 Is the performance visibly improving? Metrics M10 (Current average cycle time/Baseline average cycle time)*100 54 Combining GQM with Process Maturity 55 Summary 56 References • Software Metrics: A Rigorous & Practical Approach, by Norman E. Fenton and Shari L. Pfleeger, 2nd Edition, PWS Publishing Company, 1997(Chapter 3.2.13.2.2 ) 57 ... relative to a particular environment • Objects of measurement are – Products – Processes – Resources 15 • Processes – Software? ?related activities normally associated with time; E.g., specifying, designing, testing, ... AT&T Goals, Questions, and Metrics for Inspection? ?Process? ? Plan Goal Plan Questions How much does the inspection process? ?cost? Metrics How much calendar time does the inspection? ?process? ?take? Average effort per ... What is the status of the inspection? ?process? Total KLOC inspected 34 AT&T Goals, Questions, and Metrics for Inspection? ?Process? ? Improve Goal Improve Questions How effective is the inspection process? What is the productivity of the