VALUE-BASED PROJECT MANAGEMENT METRICS 1
5.5 THE NEED FOR EFFECTIVE MEASUREMENT TECHNIQUES
Some people believe that a customer’s greatest interest is quality. In other words, “Quality comes first! While that may seem to be true on the surface, the customer generally does not expect to pay an extraordinary amount of money just for high quality. Quality is just one component in the value equation. Value is significantly more than just quality.
When customers agree to a contract with a contractor/supplier for a deliverable, the customer is actually looking for the value in the deliverable.
The customer’s definition of success is “value achieved.”
Unfortunately, unpleasant things can happen when the project manager’s definition of success is the achievement of the deliverable (and possibly the triple constraint) and the customer’s definition of success is value. This is par- ticularly true when customers want value and you, as the contractor, focus on the profit margins of your projects.
Postulate #7 is a summation of Postulates #1 through #6. Perhaps the standard definition of success using just the triple constraints should be modified to include a business component such as value, or even be replaced by a more specific definition of value.
Sometimes the value of a project can change over time, and the project manager may not recognize that these changes have occurred. Failure to establish value expectations or lack of value in a deliverable can result from:
◾ Market unpredictability
◾ Market demand that has changed, thus changing constraints and assumptions
◾ Technology advances or inability to achieve functionality
◾ Critical resources were that were not available or resources who lacked the necessary skills
Establishing value metrics early on can identify if a project should be canceled. The ear- lier the project is canceled, the quicker we can assign the resources to those projects that have a higher perceived value and probability of suc- cess. Unfortunately early warning signs are not always present to indicate that the value will not be achieved. The most difficult metrics to establish are value-driven metrics.
5.5 THE NEED FOR EFFECTIVE MEASUREMENT TECHNIQUES
Selecting metrics and KPIs are not that difficult provided they can be mea- sured. This is the major obstacle with the value-driven metrics. On the surface, they look easy to measure, but there are complexities. Table 5-3 illustrates some of the metrics that are often treated as value-driven KPIs.
TIP Degradation in value metrics is a clear indi- cation that the project is in trouble and that it may be canceled.
Traditionally, business plans identified the benefits expected from the project and the benefits were the criteria for project selection. Today, port- folio management techniques require identification of the value as well as the benefits. However, conversion from benefits to value is not easy.6
There are shortcomings in the conversion process that can make the conversion difficult. Figure 5-4 illustrates several common shortcomings.
There are other shortcomings with the measurement of KPIs. KPI are metrics for assessing value. With traditional project management, metrics are established by the enterprise project management methodology and fixed for the duration of the project’s life cycle. With value-driven project management, however, metrics can change from project to project, during a life cycle phase and over time because of:
◾ The way the company defines value internally
◾ The way the customer and contractor jointly define success and value at project initiation
◾ The way the customer and contractor come to an agreement at project initiation as to what metrics should be used on a given project
◾ New or updated versions of tracking software
◾ Improvements to the enterprise project management methodology and the accompanying project management information system
◾ Changes in the enterprise environmental factors TABLE 5-3 Measuring Value
VALUE METRIC MEASUREMENT
Profitability Easy
Customer Satisfaction Hard
Goodwill Hard
Penetrate New Markets Easy
Develop New Technology Moderate
Technology Transfer Moderate
Reputation Hard
Stabilize Work Force Easy
Utilize Unused Capacity Easy
6. For additional information on the complexities of conversion, see Jack J. Phillips, Timothy W. Bothell, and G. Lynne Snead, The Project Management Scorecard, Oxford, UK: Butterworth Heinemann, An Imprint of Elsevier, 2002, Chapter 13.
147
5.5 THE NEED FOR EFFECTIVE MEASUREMENT TECHNIQUES
Even with the best possible metrics, measuring value can be difficult.
Some values are easy to measure, while others are more difficult. The easy value metrics to measure are often called soft or tangible value metrics, whereas the hard values are often considered as intangible value metrics.
Table 5-4 illustrates some of the easy and hard value metrics to measure.
Table 5-5 shows some of the problems associated with measuring both hard and soft value metrics
Figure 5-4 Shortcomings
Shortcomings Measurement
At Too High A Level
Failure To Measure Changes
No Legitimate Methods Available Too Many Assumptions
Must Be Made Validity of The Assumptions Questionable
Wrong People Doing The Measurement
TABLE 5-4 Typical Financial Value Metrics
EASY (SOFT/TANGIBLE) VALUE METRICS
HARD (INTANGIBLE) VALUE METRICS ROI Calculators
Net Present Value (NPV) Internal Rate of return (IRR) Cash Flow
Payback Period Profitability Market Share
Stockholder Satisfaction Stakeholder Satisfaction Customer Satisfaction Employee Retention Brand Loyalty Time-To-Market Business Relationships Safety
Reliability Goodwill Image
The intangible elements are now considered by some to be more important than tangible elements. This appears to be happening on IT proj- ects where executives are giving significantly more attention to intangible values. The critical issue with intangible values is not necessarily in the end result, but in the way that the intangibles were calculated.7
Tangible values are usually expressed quantitatively where as intan- gible values may be expressed through a qualitative assessment. There are three schools of thought for value measurement:
◾ School #1: The only thing that is important is ROI.
◾ School #2: ROI can never be calculated effectively; only the intangibles are what are important.
◾ School #3: If you cannot measure it, then it does not matter.
The three schools of thought appear to be an all-or-nothing approach where value is either 100% quantitative or 100% qualitative. The best approach is most likely a compromise between a quantitative and qualita- tive assessment of value. It may be necessary to establish an effective range, as show in Figure 5-5, which is a compromise among the three schools of thought. The effective range can expand or contract.
The timing of value measurement is absolutely critical. During the life cycle of a project, it may be necessary to switch back and forth from qualita- tive to quantitative assessment of the metric and, as stated previously, the actual metrics or KPIs may then be subject to change. Certain critical ques- tions must be addressed:
◾ When or how far along the project life cycle can we establish concrete metrics, assuming it can be done at all?
◾ Can value be simply perceived and therefore no value metrics are required?
TABLE 5-5 Problems with Measuring Value Metrics
EASY (SOFT/TANGIBLE) VALUE METRICS
HARD (INTANGIBLE) VALUE METRICS Assumptions are often not fully disclosed
and can affect decision-making Measurement is very generic Measurement never meaningfully captures the correct data
Value is almost always based upon subjective-type attributes of the person doing the measurement
It is more of an art than a science Limited models are available to perform measurement
7. For additional information on the complexities of measuring intangibles, see Jack J.
Phillips, Timothy W. Bothell, and G. Lynne Snead, The Project Management Scorecard, Oxford, UK: Butterworth Heinemann, An Imprint of Elsevier, 2002, Chapter 10. The authors empha- size that the true impact on a business must be measured in business units.
149
5.5 THE NEED FOR EFFECTIVE MEASUREMENT TECHNIQUES
◾ Even if we have value metrics, are they concrete enough to reasonably predict actual value?
◾ Will we be forced to use value-driven project management metrics on all projects or are there some projects where this approach is not necessary?
◾ Well-defined versus ill-defined
◾ Strategic versus tactical
◾ Internal versus external
◾ Can we develop a criterion for when to use value-driven project manage- ment, or should we use it on all projects but at a lower intensity level?
For some projects, using metrics to assess value at project closure may be difficult. We must establish a time frame for how long we are willing to wait to measure the final or real value or benefits from a project. This is particularly important if the actual value cannot be identified until some time after the project has been completed. Therefore, it may not be possible to appraise the success of a project at closure if the true economic values cannot be realized until some time in the future.
Some practitioners of value measurement question whether value mea- surement is better using boundary boxes instead of life cycle phases. For value-driven projects, the potential problems with life cycle phase metrics include:
◾ Metrics can change between phases and even during a phase.
◾ Inability to account for changes in the enterprise environmental factors.
◾ Focus may be on the value at the end of the phase rather than the value at the end of the project.
◾ Team members may get frustrated not being able to quantitatively calcu- late value.
Figure 5-5 Quantitative versus Qualitative Assessment
Qualitative Assessment
Quantitative Assessment
0 %
0 % 100 %
100 % Effective
Range
Boundary boxes, as shown in Figure 5-6, have some degree of similar- ity to statistic process control charts and can assist in metric measurements.
Upper and lower strategic targets for the value of the metrics are estab- lished. As long as the KPIs indicate that the project is still within the upper and lower value targets, the project’s objectives and deliverables may not undergo any scope changes or tradeoffs.
Value-driven projects must undergo value health checks to confirm that the project will make a contribution of value to the company. Value metrics, such as KPIs, indicate the current value. What is also needed is an extrapolation from the present into the future. Using traditional proj- ect management combined with the traditional enterprise project man- agement methodology, we can calculate the time at completion and the cost at completion. These are common terms that are part of earned value measurement systems. However, as stated previously, being on time and within budget is no guarantee that the perceived value will be there at proj- ect completion.
Therefore, instead of using an enterprise project management meth- odology, which focuses on earned value measurement, we may need to create a value management methodology (VMM), which stresses the value variables. With VMM, time to complete and cost to complete are still used, but we introduce a new term entitled value (or benefits) at completion.
Determination of value at completion must be done periodically through- out the project. However, periodic reevaluation of benefits and value at completion may be difficult because:
◾ There may be no reexamination process.
◾ Management is not committed and believes that the reexamination pro- cess is unreal.
◾ Management is overoptimistic and complacent with existing performance.
◾ Management is blinded by unusually high profits on other projects (misinterpretation).
◾ Management believes that the past is an indication of the future.
Figure 5-6 The Boundary Box
Project’s Objectives and Deliverables Upper Value Targets
Lower Value Targets
Value Targets
• Cash Flow
• ROI
• Delivery Dates
• Performance Metrics
151