www.it-ebooks.info www.it-ebooks.info Codermetrics www.it-ebooks.info www.it-ebooks.info Codermetrics Analytics for Improving Software Teams Jonathan Alexander Beijing • Cambridge • Farnham • Köln • Sebastopol • Tokyo www.it-ebooks.info Codermetrics by Jonathan Alexander Copyright © 2011 Jonathan Alexander. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://my.safaribooksonline.com). For more information, contact our corporate/institutional sales department: (800) 998-9938 or corporate@oreilly.com. Editors: Andy Oram and Mike Hendrickson Production Editor: Kristen Borg Proofreader: O’Reilly Production Services Indexer: John Bickelhaupt Cover Designer: Karen Montgomery Interior Designer: David Futato Illustrator: Robert Romano Printing History: August 2011: First Edition. Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly Media, Inc. Codermetrics, the image of a whitebar surgeonfish, and related trade dress are trademarks of O’Reilly Media, Inc. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and O’Reilly Media, Inc., was aware of a trademark claim, the designations have been printed in caps or initial caps. While every precaution has been taken in the preparation of this book, the publisher and authors assume no responsibility for errors or omissions, or for damages resulting from the use of the information con- tained herein. ISBN: 978-1-449-30515-4 [LSI] 1312293627 www.it-ebooks.info Table of Contents Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Part I. Concepts 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Measuring What Coders Do . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 The Purpose of Metrics 11 Metrics Are Not Grades 13 Team Dynamics 13 Connecting Activities to Goals 15 Good Metrics Shed a Light 15 Examining Assumptions 16 Timeout for an Example: The Magic Triangle (Partially) Debunked 18 Patterns, Anomalies, and Outliers 20 Peaks and Valleys 22 Ripple Effects 23 Repeatable Success 24 Understanding the Limits 25 Timeout for an Example: An Unexpected Factor in Success 25 Useful Data 27 Choosing Data 30 Obtaining Data 30 Spotters and Stat Sheets 32 Fairness and Consistency 33 Timeout for an Example: Metrics and the Skeptic 34 3. The Right Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 Questions That Metrics Can Help Answer 37 How Well Do Coders Handle Their Core Responsibilities? 38 v www.it-ebooks.info How Much Do Coders Contribute Beyond Their Core Responsibilities? 38 How Well Do Coders Interact With Others? 40 Is the Software Team Succeeding or Failing? 41 Timeout for an Example: An MVP Season 44 The Data for Metrics 45 Data on Coder Skills and Contributions 45 Data on Software Adoption, Issues, and Competition 56 Timeout for An Example: A Tale of Two Teams 62 Part II. Metrics 4. Skill Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 Input Data 69 Offensive Metrics 70 Defensive Metrics 80 Precision Metrics 86 Skill Metric Scorecards 93 Observations on Coder Types 96 Architects 96 Senior Coders 97 Junior Coders 98 5. Response Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 Input Data 101 Win Metrics 102 Loss Metrics 109 Momentum Metrics 116 Response Metric Scorecards 126 Observations on Project Types 128 Consumer Software 128 Enterprise Software 129 Developer and IT Tools 130 Cloud Services 131 6. Value Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 Input Data 133 Contribution Metrics 134 Rating Metrics 143 Value Metric Scorecards 151 Observations on Team Stages 151 Early Stage 151 Growth Stage 153 vi | Table of Contents www.it-ebooks.info Mature Stage 155 Part III. Processes 7. Metrics in Use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 Getting Started 160 Find a Sponsor 160 Create a Focus Group 160 Choose Trial Metrics 161 Conduct a Trial and Review The Findings 163 Introduce Metrics to the Team 165 Create a Metrics Storage System 166 Expand the Metrics Used 167 Establish a Forum for Discourse 168 Timeout for an Example: The Seven Percent Rule 168 Utilizing Metrics in the Development Process 170 Team Meetings 170 Project Post-Mortems 173 Mentoring 175 Establishing Team Goals and Rewards 176 Timeout for an Example: The Turn-Around 178 Using Metrics in Performance Reviews 183 Choosing Appropriate Metrics 183 Self-Evaluations and Peer Feedback 185 Peer Comparison 186 Setting Goals for Improvement 187 Promotions 187 Taking Metrics Further 189 Create a Codermetrics Council 189 Assign Analysis Projects 191 Hire a Stats Guy or Gal 192 Timeout for an Example: The Same But Different 193 8. Building Software Teams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 Goals and Profiles 201 Set Key Goals 202 Identify Constraints 202 Find Comparable Team Profiles 203 Build a Target Team Profile 208 Roles 211 Playmakers and Scorers 211 Defensive Stoppers 212 Table of Contents | vii www.it-ebooks.info Utility Players 213 Role Players 214 Backups 215 Motivators 215 Veterans and Rookies 216 Timeout for an Example: Two All-Nighters 218 Personnel 220 Recruit for Comps 221 Establish a Farm System 224 Make Trades 225 Coach the Skills You Need 226 Timeout for an Example: No Such Thing As a Perfect Team 227 9. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 A. Codermetrics Quick Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 B. Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 viii | Table of Contents www.it-ebooks.info [...]... development process As a manager of software teams, I myself am on that journey I believe the techniques in this book have helped “turn around” troubled software teams and have helped good software teams become even better Gathering metrics on a wider set of activities and outcomes isn’t the only path to success, of course, but it has worked for me, and I believe it can work for you, too Maybe you measure... ways for software development teams, then this book is designed with you in mind The ideas in this book are a departure from how metrics have been applied to software development in the past The concepts and techniques presented here are meant to help you think differently about building software teams and to help get you started on your own journey using metrics in new and better ways as part of the software. .. Fog” This is a book about coders, software development teams, metrics and patterns The ideas in this book originated a few years ago when I started to think about the makeup of software teams, both good and bad, and all the subtle contributions and unsung heroes that are a critical part of success For almost two decades now, I’ve been responsible for building and managing teams of designers, coders, and... order, although you may find specific parts of the book more useful for later review if you are putting metrics into practice Part I, “Concepts”, provides a more detailed introduction behind the thinking of codermetrics, the variety of analyses that metrics can enable, and the data that can be measured for coders and software development teams Part II, “Metrics”, is set up as a kind of metrics reference... there a rational way to measure coder skills and contributions and the way that software teams fit together? Could metrics help you improve coder self-awareness, teamwork, mentoring, and goal-setting? Could more detailed data help you make better hiring decisions, help make performance reviews fairer, and help your software teams become more successful? Whether you are a coder, team leader, or manager,... designers, coders, and testers Over this time I’ve realized that software teams, similar to sports teams, require a variety of players and skills to succeed I’ve also learned that there are patterns to success and failure that are not necessarily what I assumed before Here’s a simple pattern I’ve seen, and maybe you’ve seen it too: every successful software team I’ve been on has always had at least one person... the successes or failures of our software teams and the full range of coder skills and contributions Not as a way to rate performance, but as a way to help us better understand and foster the keys to success, and where and how we might improve In my own teams I began to experiment, and the positive results have me very encouraged that these methods could be useful for others, too This book is my attempt... sports analogies It’s not necessary, however, for you to like or understand sports to understand the concepts in this book Like all analogies, the purpose is just to help make the ideas quicker to grasp and easier to remember Personally, I think using sports analogies to discuss software teams is apt—and fun I think of software development as a team sport Software products are typically not produced... Blown Save These stats meet the criteria for good metrics presented above: namely, they are relatively easy to understand, they show us something that we otherwise wouldn’t easily track, and they relate directly to key goals for players and teams (wins and losses) You could imagine a metric similar to baseball saves, but for coders A coder could get credit for fixing critical issues at the late stages... production team, thanks, too For the feedback and reviews they provided on this book during the process, I want to thank Brian Jackson at Google, Nagaraj Nadendla at Taleo, and Ben Wu at Zuora They are all excellent leaders and managers themselves Thanks guys To my dad, thanks for giving me a love of sports, which has become a love for statistics too To my mom, thanks for encouraging me to write a . www.it-ebooks.info www.it-ebooks.info Codermetrics www.it-ebooks.info www.it-ebooks.info Codermetrics Analytics for Improving Software Teams Jonathan Alexander Beijing • Cambridge • Farnham • Köln • Sebastopol • Tokyo www.it-ebooks.info Codermetrics by. re- sponsible for building and managing teams of designers, coders, and testers. Over this time I’ve realized that software teams, similar to sports teams, require