The use of metrics and KPIs has been with us for decades, but the use of a value metric is relatively new. As such, failures in the use of this technique are still common and may include:
◾ Not forward looking; the value metric focuses on the present rather than the future
◾ Does not go beyond financial metrics and, thus, fails to consider the value in knowledge gained, organizational capability, customer satisfac- tion, and political impacts
◾ Believing that value metrics (and the results) that other companies use will be the same for your company
◾ Not considering how the client and stakeholders define value
◾ Allowing the weighting factors to change too often, to make the project’s results look better
As with any new technique, additional issues always arise. Typical questions that we are now trying to answer in regard to the use of a value metric include:
◾ What if only three of the five components can be measured, for example, in the early life cycle phases of a project?
◾ In such a case where only some components can be measured, should the weighting factors be changed or normalized to 100 percent, or left alone?
◾ Should the project be a certain percent complete before the value metric has any real meaning?
◾ Who will make decisions as to changes in the weighting factors as the project progresses through its life cycle phases?
◾ Can the measurement technique for a given component change over each life cycle phase or must it be the same throughout the project?
◾ Can we reduce the subjectivity of the process?
5.14 INDUSTRY EXAMPLES OF VALUE METRICS
This section provides examples of how various companies use value met- rics. The number of companies using value metrics is still quite small. Some of the companies surveyed could not provide accurate weighting factors because the factors can change for each project. Other companies differen- tiated between project success and product success and stated that the value attributes and weighting factors were different for each as well. (Note: The names of the companies were withheld at the request of the companies.)
In several of the examples, there are descriptions of the attributes. In most cases, the attributes of the final value metric are a composite of vari- ous KPIs as discussed previously in this chapter.
Aerospace and Defense: (Company 1)
◾ Schedule: 25% (This would result from objective data via the project’s earned value measurement system.)
◾ Cost: 25% (This would result from objective data via the project’s earned value measurement system.)
◾ Technical factor: 30% (This would be based upon Technical Performance Measures established at the beginning of the project.)
◾ Quality factor: 10% (This results from an ongoing audit of the adherence to the established quality standards and procedures.)
◾ Risk factor: 10% (This is based upon how well risk mitigation plans are implemented and followed.)
These percentages are for a generic project. Depending upon the nature of the project and the constraints imposed by the client, the percentages would be adjusted. The actual percentages would be coordinated with the client so that they would know and agree to the weighting to be used. The projects that this would apply to would result in a product or system, or both. Most of the clients in aerospace and defense are extremely sensitive to cost and schedule, which is why the percentages for those are as high as they are.
Aerospace and Defense: (Company 2)
◾ Quality: 35%
◾ Delivery: 25%
◾ Cost: 20%
◾ Technology: 5%
◾ Responsiveness: 10%
◾ General Management: 5%
Capital Projects:
◾ Revenue growth/generation: 30% (Our primary focus has been on grow- ing revenue through leveraging alternative options.)
◾ Cost efficiencies: 30% (This has a direct bottom-line impact and seen almost equal to generating revenue.)
◾ Handle/market share growth: 20% (There’s a revenue impact here as well.)
◾ Project schedule: 10% (We have many time-constrained projects because of our core business operating model. We have the natural tradeoff deci- sions; however, the first three factors are weighted more heavily.)
◾ Project cost: 10% (Generally tracked to ensure increasing costs do not overwhelm expected benefits)
IT Consulting [External Clients]: (Company 1) No percentages provided
◾ Risks
◾ Scope
◾ Resources
◾ Quality
◾ Schedule
◾ Overall status
179
5.14 INDUSTRY EXAMPLES OF VALUE METRICS
IT Consulting [External Clients]: (Company 2)
◾ Quality: 40% (as determined by the feedback from internal engagement leaders review of project deliverables before they are submitted to the cli- ent, the number of iterations before the deliverables are client-ready, and eventually, by the client’s satisfaction with the provided deliverables, and the manner in which client interactions and expectations were managed throughout the engagement, that is, did we make and keep our promises, did we offer the client a “no-surprise” experience, and did we demonstrate professionalism through effective interaction and communications?)
◾ Talent: 20% (as determined by feedback from Firm and engagement leaders as well as satisfaction of project team members in terms of how the project was planned and delivered, that is, was there a posi- tive working environment that was created; were the views and opin- ions of individual team members valued and considered; were the roles and contributions of individual team members well defined, properly communicated, and understood by all involved; did the project provide an opportunity for personal and professional growth and development, etc.?)
◾ Marketplace: 10% (as determined by the evaluation of the Firm and engagement leadership as to the extent to which the given project dem- onstrated understanding of the client and the client’s industry, as well as the extent to which the project contributed to the establishing or sup- porting the firm’s preeminence within the given service line or industry)
◾ Financial: 30% (as determined by on-time and on-budget delivery of the project, the profitability of the engagement, the achieved recovery rate or the level of applied discount to standard rates, and the ability to submit invoices and collect payment within a reasonable amount of time)
IT Consulting [External Clients]: (Company 3)
◾ Customer Satisfaction: 30% (their perception on how well the project is going and satisfaction with the team and solution)
◾ Budget: 20%
◾ Schedule: 10%
◾ Solution Deployed: 20% (solution is being used by the customer in pro- duction and providing value)
◾ Support Issues: 10%
◾ Opportunity Generation: 10% (a successful project for us would also generate additional opportunities with the customer—difficult to mea- sure until well after project completion)
These factors would apply to all industries and solutions. The only dif- ferences are that the Solution Deployed is not applicable in certain project types (such as a Health Check), and Opportunity Generation is only appli- cable when looking back in time after a project is completed.
IT Consulting [External Clients]: (Company 4)
◾ Customer Satisfaction/Conditions of Satisfaction: 30%
◾ Manage Expectations/Communication: 20%
◾ Usability/Performance: 20%
◾ Quality: 20%
◾ Cost: 10%
It is less about time and cost now and more about can/will the prod- uct/service be used and is the client happy with the delivery of the final product/service?
IT Consulting [External Clients]: (Company 5)
These value metric attributes are primarily for consulting in healthcare.
◾ Quality: 25%
◾ Cost: 20%
◾ Durations/Timeliness: 15%
◾ Resource Utilization: 10%
◾ Incorporation of Processes (clinical, technical, business focused): 30%
IT Consulting [External Clients]: (Company 6)
Similar to Company 5, this company also provided IT consulting services to healthcare but did not provide a breakdown of the value metric. They believe that the value metric (whether specifically outlined or not on a project) is the true reason why these projects continue to be funded beyond their schedule and budget.
IT Consulting [Internal]:
◾ Scope: 25%
◾ Project Client Satisfaction: 22.5%
◾ Schedule: 17.5%
◾ Budget: 17.5%
◾ Quality: 17.5%
Software Development [Internal]: No percentages provided
◾ Code: number of lines of code
◾ Language understandability: language and/or code is easy to understand and read
◾ Movability/immovability: The ease by which information can be moved
◾ Complexity: loops, conditional statements, etc . . .
◾ Math complexity: time and money to execute algorithms
◾ Input/output understandability: How difficult is it to understand the program?
Telecommunications: (Company 1)
◾ Financial: 35%
◾ Quality/Customer Satisfaction: 35%
181
5.14 INDUSTRY EXAMPLES OF VALUE METRICS
◾ Process Adherence: 15%
◾ Teamwork: 15%
Telecommunications: (Company 2) No percentages provided
◾ Customer satisfaction
◾ Employee satisfaction
◾ Quality
◾ Financial
◾ Cost
New Product Development:
◾ Features/Functions: 35% (This is where the company believes it can dif- ferentiate itself from its competitors.)
◾ Time to market: 25%
◾ Quality: 25%
◾ Cost: 15%
Automotive Suppliers:
◾ Quality: 100%
◾ Cost: 100%
◾ Safety: 100%
◾ Timing: 100%
It is interesting to note that in this company there were four value met- rics rather than just one, and the company believed that the four value metrics could not be combined into a single value metric. This is why 100 percent is assigned to each. In the auto industry, being less than 100 percent on each value metric could delay the launch of a product and create finan- cial problems for the client and all of the suppliers.
Global Consulting: ( Company 1) [Not industry specific and no weights]
This company’s observations are that soft skills and personal attributes are some of the key factors affecting the outcome of projects and very little association has so far been made between that and project failure.
This is why some of the soft skills are part of their value metric. The challenge is in quantifying these soft skills factors for Customer Value Management. This will vary for each project, which is why percentages are not provided.
◾ Management:(consideration given to quick resolution of issues, mini- mal time wasted, minimal recycling of ideas, timely escalations)
◾ Communications/Relationship Building: (verbal and written status reports, weekly meetings, and the like. agreed to at the start of the project)
◾ Competency:(impact and influence as well as performance and knowl- edge of project management)
◾ Flexibility and Commitment: (balancing clients’ requests and require- ments with that of suppliers and third-party contractors—ability to mon- itor and control to bring optimum results)
◾ Quality: (The quality definition would vary based on project deliver- ables. This would be influenced by the industry.)
◾ Usability:(how much of what is implemented can give immediate added value—this is on the premise that few organizations progress beyond the implemented subset under two years by which time the original users would have moved on and their successors would not have had the vision to move the systems to the next level. On the contrary the reverse happens, as less functionality is utilized by the inheritors of the systems.)
◾ Delivery Strategy: (this speaks directly to the strategy or approach for implementation of the solution offered to the client—the solution on paper could have seemed to meet the client’s need but if not executed optimally, can cause added pains rather than gains. This speaks directly to governance approaches, production of specified deliverables, perform- ing tasks on schedule, and the like.)
◾ Customer Focus: (to what extent does the PM team seek the interest of the customer/client)
◾ HSE: (the satisfaction level with the Project Team’s compliance with health, safety, and environmental policies while performing duties at the customer’s site)
Global Consulting: ( Company 2) [Not Industry Specific and no weights]
◾ Profitability
◾ Schedule
◾ Impact/Result
◾ Customer Satisfaction
◾ Safety
◾ Erosion
Erosion is the value of work performed in excess of what was originally estimated in terms of effort or duration, and not recovered through proj- ect change management procedures. In simple terms, erosion is the differ- ence between billable work (that, which has been estimated, proposed and accepted by the customer) and nonbillable work (that, which has not been estimated, proposed, and accepted by the customer).