Because of these measurement techniques, companies are now tracking a dozen or more metrics on projects. While this sounds good, it has cre- ated the additional problem of potential information overload. Having too
11. For a description of several of these techniques, see Douglas W. Hubbard, How to Measure Anything, Hoboken, NJ: John Wiley & Sons Publishers, 2007.
167
5.10 SELECTING THE RIGHT METRICS
many performance metrics may provide the viewers with more informa- tion than they actually need and, therefore they may not be able to discern the true status or what information is really important. It may be hard to ascertain what is important and what is not, especially if decisions must be made. Providing too few metrics can make it difficult for the viewers to make informed decisions. There is also a cost associated with metric mea- surement, and we must determine if the benefits of using this many metrics outweigh the costs of measurement. Cost is important because we tend to select more metrics than we actually need.
There are three categories of metrics:
◾ Traditional metrics: These metrics are used for measuring the perfor- mance of the applied project management discipline more so than the results of the project and how well we are managing according to the predetermined baselines. (e.g., cost variance and schedule variance)
◾ Key performance indicators (KPIs): These are the few selected metrics that can be used to track and predict whether the project will be a suc- cess. These KPIs are used to validate that the critical success factors (CSFs) defined at the initiation of the project are being met. (e.g., time-at- completion, cost-at-completion, and customer satisfaction surveys)
◾ Value metrics: These are special metrics that are used to indicate whether the stakeholders’ expectations of project value are or will be met. Value metrics can be a combination of traditional metrics and KPIs. (value-at- completion and time to achieve full value)
Each type of metric has a primary audience, as shown in Table 5-9.
There can be three information systems on a project:
◾ One for the project manager
◾ One for the project manager’s superior or parent company
◾ One for the stakeholders and the client
There can be a different set of metrics and KPIs for each of these infor- mation systems.
TABLE 5-9 Audiences for Various Metrics
TYPE OF METRIC AUDIENCE
Traditional metrics Primarily the project manager and the team, but may include the internal sponsor(s) as well Key performance indicators Some internal usage but mainly used for status
reporting for the client and the stakeholders Value metrics Can be useful for everyone but primarily for the
client
Traditional metrics, such baselines as the cost, scope ,and sched- ule, track and can provide information on how well we are performing according to the processes in each Knowledge Area or Domain Area in the PMBOK® Guide. Project manager must be careful not to micromanage their project and establish 40 to 50 metrics.
Typical metrics may include:
◾ Number of assigned versus planned resources
◾ Quality of assigned versus planned resources
◾ Project complexity factor
◾ Customer satisfaction rating
◾ Number of critical constraints
◾ Number of cost revisions
◾ Number of critical assumptions
◾ Number of unstaffed hours
◾ Percent of total labor hours on overtime
◾ Cost variance
◾ Schedule performance index
◾ Cost performance index
This is obviously not an all-inclusive list. These metrics may have some importance for the project manager but not necessarily the same degree of importance for the client and the stakeholders.
Clients and stakeholders are interested in critical metrics or KPIs. These chosen few metrics are reported to the client and stakeholders and provide an indication of whether or not success is possible; however, they do not necessarily identify if the desired value will be achieved. The number of KPIs is usually determined by the amount of real estate on a computer screen. Most dashboards can display between six and ten icons or images where the information can be readily seen with reasonable ease.
To understand what a KPI means requires a dissection of each of the terms:
◾ Key: a major contributor to success or failure
◾ Performance: measurable, quantifiable, adjustable and controllable elements
◾ Indicator: reasonable representation of present and future performance Obviously, not all metrics are KPIs. There are six attributes of a KPI, and these attributes are important when identifying and selecting the KPIs.
◾ Predictive: able to predict the future of this trend
◾ Measurable: can be expressed quantitatively
◾ Actionable: triggers changes that may be necessary
◾ Relevant: the KPI is directly related to the success or failure of the project
169
5.10 SELECTING THE RIGHT METRICS
◾ Automated: reporting minimizes the chance of human error
◾ Few in number: only what is necessary
Applying these six attributes to traditional metrics is highly subjective and will be based upon the agreed -upon definition of success, the criti- cal success factors (CSFs) that were selected, and possibly the whims of the stakeholders. There can be a different set of KPIs for each stakeholder based upon each stakeholder’s definition of project success and final proj- ect value. This could significantly increase the costs of measurement and reporting, especially if each stakeholder requires a different dashboard with different metrics.
Previously we identified twelve possible metrics that could be used on projects, but how many of those 12 are actually regarded as a KPI? If we apply the first five of the six KPI attributes identified above, we could end up with the representation as shown in Table 5-10.
Only six of the twelve metrics (1, 3, 4, 8, 11, and 12) are regarded as a KPI and, once again, this is often a highly subjective selection process. In this example, these would be the critical metrics that would be shown on the project dashboard and could be a necessity for informed decision mak- ing. The other metrics can still be used, but the reader may need to “drill down” on the screens to get access to the traditional metrics.
TABLE 5-10 Selecting the KPIs
PREDICTIVE MEASURABLE ACTIONABLE RELEVANT AUTOMATED 1. # of assigned vs.
planned res.
✓ ✓ ✓ ✓ ✓
2. Quality of assigned vs.
planned res.
✓ ✓ ✓
3. Project complexity factor ✓ ✓ ✓ ✓ ✓
4. Customer satisfaction rating ✓ ✓ ✓ ✓ ✓
5. # of critical constraints ✓ ✓ ✓
6. # of cost revisions ✓ ✓ ✓ ✓
7. # of critical assumptions ✓ ✓ ✓
8. # of unstaffed hours ✓ ✓ ✓ ✓ ✓
9. % of overtime labor hours ✓ ✓ ✓
10. Cost variance ✓ ✓
11. Schedule performance index ✓ ✓ ✓ ✓ ✓
12. Cost performance index ✓ ✓ ✓ ✓ ✓