The Project Management Scorecard Measuring the Success of Project Management Solutions Jack J. Phillips, Ph.D. Timothy W. Bothell, Ph.D. G. Lynne Snead Amsterdam Boston London New York Oxford Paris San Diego San Francisco Singapore Sydney Tokyo An Imprint of Elsevier Science Copyright © 2002 by Elsevier Science (USA) Butterworth–Heinemann is an imprint of Elsevier Science All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher. Chapters 1–3 and the Appendix, as well as the graphs, models, trademarks, processes, and methodology used throughout this book are the property of Franklin Covey Co., © 2002 Franklin Covey Co., and are used herein under license. Recognizing the importance of preserving what has been written, Elsevier Science prints its books on acid-free paper whenever possible. Library of Congress Cataloging-in-Publication Data Phillips, Jack J., 1945– The project management scorecard : measuring the success of project manage- ment solutions / Jack J. Phillips, Timothy W. Bothell, G. Lynne Snead. p. cm.—(Improving human performance series) Includes bibliographical references and index. ISBN 0-7506-7449-0 (alk. paper) 1. Project management. 2. Project management—Evaluation. I. Bothell, Timothy W. II. Snead, G. Lynne. III. Title. IV. Series. HD69.P75 P49 2001 658.4'04—dc21 2001052837 British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library. The publisher offers special discounts on bulk orders of this book. For information, please contact: Manager of Special Sales Elsevier Science 225 Wildwood Avenue Woburn, MA 01801–2041 Tel: 781-904-2500 Fax: 781-904-2620 For information on all Butterworth–Heinemann publications available, contact our World Wide Web home page at: http://www.bh.com 10 9 8 7 6 5 4 3 2 1 Printed in the United States of America Contents v Preface xi Interest in scorecard measures xii Needed: an effective project management scorecared xii Why this book at this time? xiii Target audience xiii Structure of this book xiv Chapter descriptions xiv Acknowledgments xix Part I: Setting the Stage Project Management Issues and Challenges 3 Reasons for failed or poorly managed projects6 What is the cost of a failed or poorly managed project?6 Success factors8 Strategic planning pyramid 9 Final thoughts 10 References 10 The Project Management Process 12 Process overview 12 Following the process 13 Final thoughts 23 References 23 Further reading 23 Project Management Solutions 33 Project management solutions - past and future 33 Three stages of project management effictiveness implementation 34 Changing corporate cultures: readiness for project management solutions 34 Three types of solutions: process, tools and training 34 Mappy solutions to four levels of project management effectiveness 34 A successful approach - inside out and outside in, simultaneously 34 Project management office 43 Final thoughts 43 Reference 43 The Project Management Scorecard 49 What can happen without the scorecard? 51 Potential steps to build the project management scorecard 51 Preliminary evaluation information 51 Evaluation planning 51 ROI analysis plan 51 Final thoughts 62 References 62 Part II: The Seven Measures How to Measure Reaction and Satisfaction 77 Sources of data 77 Data collection methods 79 Using reaction and satisfaction data 81 Shortcut ways to measure reaction and satisfaction 94 Final thoughts 94 Further reading 94 How to Measure Skill and Knowledge Changes During the Project 98 Measuring learning with formal tests 98 Measuring learning with stimulation 98 Measuring learning with less structured activities 98 Administrative issues 104 Using learning data 107 Final thoughts 107 Further reading 107 How to Measure Implementation, Application, and Progress 110 Why measure application and implementation? 110 Key issues 110 Using questionnaires to measure application and implementation 112 Using interviews and focus groups to measure implementation and application 114 Observing project team members on the job to measure implementation and application 125 Using action plans and follow-up assignments to measure implementation and application 126 Using performance contracts to measure implementation and application 129 Shortcut ways to measure application and implementation 136 Final thoughts 138 Further reading 138 How to Capture Business Impact Data 140 Why measure business impact? 140 Types of data 140 Monitoring business performance data 141 Using action plans to develop business impace data 150 Using questionnaires to collect business impace measures 157 ROI analysis 157 Selecting the appropriate method for each level 159 How the credibility of data is influenced 160 Shortcut ways to capture business impact data 163 Final thoughts 165 Further reading 166 How to Calculate and Interpret ROI 168 Basic issues 168 Benefit/cost ratio 168 ROI formula 169 Other ROI measures 172 Scorecard issues 174 Final thoughts 178 Further reading 179 Identifying Intangible Measures of a Project Management Solution 180 Why identify intangibles? 180 Measuring employee satisfaction 184 Measuring employee withdrawal 184 Measuring customer service 186 Measuring team effectiveness 187 Final thoughts 188 Further reading 189 Monitoring the True Costs of the Project Solution 190 Why monitor project solution costs? 190 How to develop costs 191 Cost-tracking issues 192 Major cost categories 194 Cost accumulation and estimation 198 Final thoughts 206 Reference 206 Further reading 206 Part III: Key Issues with the Measures How to Isolate the Effects of Project Management Solutions 209 Why the concern with this issue? 209 Preliminary issues 210 Use of control groups 212 Trend-line analysis 214 Forecasting methods 215 Project team members’ estimte of impact 215 Customer input on project management solution impact 221 Expert estimation of project solution impact 222 Calculating the impact of other factors 222 Using the techniques 223 Shortcut ways to isolate the effects of the project management solution 224 Final thoughts 224 References 225 Further reading 225 How to Convert Business Measures to Monetary Values 226 Why convert data to monetary values? 226 The five key steps to converting data to monetary values 227 How does it work? 228 Techniques for converting data to monetary values 228 Converting output data 229 Calculating the standard cost of quality 231 Converting employee time using compensation 233 Using historical costs from records 234 Using input from internal and external experts 234 Using values from external databases 235 Linking with other measures 236 Using estimates from project team members 238 Using estimates from the management team 239 Selecting the techniques and finalizing the values 240 Shortcut ways to convert data to monetary values 243 Final thoughts 243 References 244 Further reading 244 Part IV: Challenges Forecasting ROI: How to Build a Business Case for the Project Management Solution 249 Why forecast ROI? 249 The trade-offs of forecasting 251 Pre-project ROI forecasting 254 Forecasting with a pilot program 261 Forecasting ROI with reaction data 262 Final thoughts 267 Further reading 268 How to Provide Project Feedback and Communicate Results to the Client 269 Why be concerned about communicating results? 269 Principles of communicating results 271 A model for communicating results 271 Analyzing the need for communication 275 Planning the communication 277 Selecting the audience for communications 280 Developing the information: the impact study 280 Selecting the communication media 287 Communicating the information 291 Analyzing reactions to communication 291 Shortcut ways to provide feedback and communicate with stakeholders 298 Final thoughts 299 Further reading 299 Overcoming Resistance and Barriers to the Project Management Scorecard 301 Why by concerned about overcoming resistance? 301 The approach to overcoming resistance 301 Developing roles and responsibilities 305 Establishing goals and plans 308 Revising/developing policies and guidelines 310 Preparing the project team 311 Initiating the scorecard project 312 Preparing the management team 314 Removing obstacles 316 Monitoring progress 316 Shortcut ways to make the scorecard process work 316 Final thoughts 317 Further reading 317 Appendix: Establishing an Effective PM Culture 319 Index 335 [...]... valuable insights into project management from this book STRUCTURE OF THIS BOOK This book is written to provide the reader with a brief introduction to the project management process and an overview of the entire PREFACE xv process needed to develop the project management scorecard Chapters in the book will explain the methodology required for each step in the project management scorecard The book provides... decision on the project Here, the detailed project plan will specify what work needs to be done, what resources are needed to do it, and what the time requirement is for the project This is the level of detail necessary for the final decision to continue to the Implement stage 14 THE PR OJECT M ANAGEMENT SCORECARD Figure 2-2 Checkpoints within the project management process The purpose of the decision... a project looks like will lead to a better-managed project, which generally leads to a better ROI Likewise, a project that was halted early rather than later in the project management process will always save organizations time, money, and resources, which will improve the ROI of the organizations’ project management processes THE PROJECT MANAGEMENT PROCESS FOLLOWING THE 15 PROCESS Visualize The Project. .. triple, the costs of a project Thus, what is the savings for managing a project well? The savings could be tremendous Determining whether an organization is working on the right projects at the right time is a key success factor The following questions may help determine whether it is the right project at the right time: I Are resources being wasted in the organization by work being done on low-priority projects?... controlling the project, and making changes and adjustments The final stage in the process, Close, looks at bringing a project through to successful completion, finalizing project documentation, and evaluFigure 2-1 ating the return on investment of the project Project manageWithin the process, there are two checkpoints to ment process ensure that the project is headed in the right direction (according to the. .. improve the management of projects This is also the reason many organizations want to justify the efforts and resources for improving project management using a balanced set of measures that considers the effectiveness of projects from all angles As a result, organizations need a Project Management Scorecard. ” Project management is an ideal process to evaluate and measure The reasons for this are: I Projects... for project management improvement Across many well-managed projects, there are common success factors Quality projects include a uniform process, an effective tool set, proper training, clear roles and responsibilities, and other factors This chapter reviews these important factors xvi THE PR OJECT M ANAGEMENT SCORECARD Chapter 4: The Project Management Scorecard Chapter 4 describes the project management. .. mentality rather than the project mentality is that the step-by-step process builds solutions that avoid the most common reasons for project failure While the project management process remains a repeatable formula, the same is not true of the tools used to help manage projects These vary tremendously and can be customized according to availability, needs, styles, and preferences Project management. .. the outcomes of the project and effectively managing the project Understanding the organizational results impacted by the project helps project managers visualize the evaluation that should be implemented throughout the project The Good News As Mahan Khalsa says in his book, Let’s Get Real or Let’s Not Play: The Demise of Dysfunctional Selling and the Advent of Helping Clients Succeed, the good news... ultimately determine if the project is a success or a failure These often include the customer, boss, project sponsor or owner, as well as the project manager and key members of the project team If the members of this group (our clients) have different visions of the end result, the project is doomed to fail before it ever begins Ensuring that this group has the same vision of the end result requires . entire xiv THE PROJECT MANAGEMENT SCORECARD process needed to develop the project management scorecard. Chapters in the book will explain the methodology required for each step in the project management. responsibilities, and other fac- tors. This chapter reviews these important factors. PREFACE xv Chapter 4: The Project Management Scorecard Chapter 4 describes the project management scorecard and how organizations. of project management. The general trend toward xii THE PROJECT MANAGEMENT SCORECARD accountability with all staff support groups is causing some managers to measure their own contribution to the