Lecture Software process improvement: Lesson 30A provide students with knowledge about: software process benchmarks; macroeconomic benchmarks; economic benchmarks; corporate IT benchmarks; customer satisfaction benchmarks; project-level benchmarks;... Please refer to the detailed content of the lecture!
Software Process Benchmarks Lecture # 30A 1 Benchmarks • Benchmarks – – – – – Standard Yardstick Target Scale Point of reference 2 Software Benchmarks • Benchmarks compare a company against industry norms 3 Software Benchmarks • Benchmarks collect quantitative and qualitative data on a number of important topics, including investments, staffing levels, development schedules, staff efforts, costs, quality, and customer satisfaction • All of these factors are related to software process improvement initiatives 4 Software Benchmarks • Benchmark studies often include supplemental information on specific tool suites, programming languages, and formal methods utilized • Thus, benchmark studies and assessment studies overlap to a degree 5 Software Benchmarks • Benchmarks are primarily comparisons between a specific company and other companies in the same kind of business 6 Software Benchmarks • Benchmark studies or comparisons of quantitative data are older than the software industry. Indeed, comparisons between the companies have been taking place for centuries. E.g., studies of market shares, staff compensation levels, executive pay, and customer satisfaction have long been performed 7 Software Benchmark • Software benchmark is a formal comparison of software methods and results against those of other organizations 8 Software Benchmarks • Software benchmarks can be traced back to the 1960s, when studies of data center performance and downtimes started to be performed • These were soon followed by corporate level studies of investments in information technology, compensation studies of software personnel, and studies of software schedules, cost, and process 9 Software Benchmarks • Within the software industry, there are a number of levels of benchmark studies that have been noted to occur • It would be a good idea to consider the various kinds of benchmarks 10 10 Following slide to be inserted Basic Sequence of Software Benchmarks 24 24 Basic Sequence of Software Benchmarks 25 • Benchmarks are even more effective than assessments in causing companies to start software process improvement programs • If a company is well below average ranges in productivity and quality, this situation presents tangible data that executives can understand easily 26 26 • For example, the virtues of moving up from an SEI CMMI level 1 to a level 3 might be abstract and difficult to explain, but the need to improve productivity rates from five function points per staff month to ten to reach industry norms is fairly unambiguous 27 27 • Accurate benchmarking of software projects was difficult for many years due to problems with the LOC software metric, which was unreliable for studies involving multiple programming languages • The advent of the function point metric has opened a door to more accurate benchmark studies of software projects 28 28 • Benchmark studies can explore a number of different topics • Very interesting software benchmarks can be carried out at the project level. These projectlevel benchmarks can generate very useful information. However, projectlevel benchmarks require a great deal of care to ensure an applestoapples comparisons 29 29 Most FAQs – 1 • What are bestinclass productivity rates in our industry? • What are bestinclass development costs per function point in our industry? • What are bestinclass development schedules? • What are bestinclass maintenance assignment scopes in our industry? 30 30 Most FAQs – 2 • What are bestinclass quality levels in our industry? • What are the best development processes for software like ours? • What are the best development tools for software like ours? 31 31 Most FAQs – 3 • What does it cost to move from SEI CMMI level 1 to SEI CMMI level 3? • What are the differences in results between SEI CMMI level 1 and CMMI level 3? 32 32 • As can be seen from these very common questions, the most interesting topics from benchmarks are those associated with best practices and bestinclass results • Averages are interesting and useful, but benchmark studies are often commissioned by companies with goals that are to be much better than average 33 33 • For benchmarks to be useful, or even for the comparisons to be valid, the projects that are compared during the benchmark have to be similar in size and in nature • For benchmarking purposes, it is not a normal practice to compare unlike software applications, such as comparing the productivity level of a small Web applet with the productivity level of a large military 34 34 system Following slide to be inserted Benchmarking Overview 35 35 Benchmarking Overview MEASURING ASSESSING IMPROVING QUANTITATIVE DATA Size Effort Schedule Documentation Defects Productivity Rates Quality Levels QUALITATIVE DATA Personnel Processes Technology Environment Where You Are Best Case Models Project Profiles Department Models Why You Are How You Should Be 36 36 Summary 37 37 References • Software Assessments, Benchmarks, and Best Practices by Capers Jones (Chapter 3) 38 38 ... and customer satisfaction have long been performed 7 Software? ?Benchmark • Software? ?benchmark is a formal comparison of? ?software? ?methods and results against those of other organizations 8 Software? ?Benchmarks • Software? ?benchmarks can be traced back to ... level studies of investments in information technology, compensation studies of software? ?personnel, and studies of? ?software? ? schedules, cost, and? ?process 9 Software? ?Benchmarks • Within the? ?software? ?industry, there are a number of levels of benchmark studies that ... Basic Sequence of? ?Software? ? Benchmarks 24 24 Basic Sequence of? ?Software? ? Benchmarks 25 • Benchmarks are even more effective than assessments in causing companies to start software? ?process? ?improvement programs