Measuring Information Systems Delivery Quality Evan W. Duggan University of Alabama, USA Han Reichgelt Georgia Southern University, USA Hershey • London • Melbourne • Singapore IDEA GROUP PUBLISHING Acquisitions Editor: Michelle Potter Development Editor: Kristin Roth Senior Managing Editor: Amanda Appicello Managing Editor: Jennifer Neidig Copy Editor: April Schmidt Typesetter: Cindy Consonery Cover Design: Lisa Tosheff Printed at: Yurchak Printing Inc. Published in the United States of America by Idea Group Publishing (an imprint of Idea Group Inc.) 701 E. Chocolate Avenue Hershey PA 17033 Tel: 717-533-8845 Fax: 717-533-8661 E-mail: cust@idea-group.com Web site: http://www.idea-group.com and in the United Kingdom by Idea Group Publishing (an imprint of Idea Group Inc.) 3 Henrietta Street Covent Garden London WC2E 8LU Tel: 44 20 7240 0856 Fax: 44 20 7379 0609 Web site: http://www.eurospanonline.com Copyright © 2006 by Idea Group Inc. All rights reserved. No part of this book may be reproduced, stored or distributed in any form or by any means, electronic or mechanical, including photocopying, without written permission from the publisher. Product or company names used in this book are for identification purposes only. Inclusion of the names of the products or companies does not indicate a claim of ownership by IGI of the trademark or registered trademark. Library of Congress Cataloging-in-Publication Data Measuring information systems delivery quality / Evan W. Duggan and Han Reichgelt, editors. p. cm. Summary: "The book provides analyses and explains some of the contradictions and apparent paradoxes of many information systems quality perspectives"--Provided by publisher. Includes bibliographical references and index. ISBN 1-59140-857-1 (hardcover) -- ISBN 1-59140-858-X (softcover) -- ISBN 1-59140-859-8 (ebook) 1. Management information systems--Evaluation. 2. Information technology--Evaluation. I. Duggan, Evan W. II. Reichgelt, Han. T58.64.M435 2006 658.4'038--dc22 2006003552 British Cataloguing in Publication Data A Cataloguing in Publication record for this book is available from the British Library. All work contributed to this book is new, previously-unpublished material. The views expressed in this book are those of the authors, but not necessarily of the publisher. Measuring Information Systems Delivery Quality Table of Contents Foreword . vi Joseph A. De Feo, Juran Institute Inc., USA Preface . viii S ECTION I: I NTRODUCTION AND Overview OF Q UALITY C ONCEPTS AND D IMENSIONS Chapter I. The Panorama of Information Systems Quality 1 Evan W. Duggan, University of Alabama, USA Han Reichgelt, Georgia Southern University, USA Chapter II. An Overview of Software Quality Concepts and Management Issues 28 Alain April, École de technologie supérieure, Québec, Canada Claude Y. Laporte, École de technologie supérieure, Québec, Canada Chapter III. The Different Views of Software Quality . 55 Bernard Wong, University of Technology, Australia S ECTION II: Q UALITY I N THE E ARLY S TAGES OF IS D ELIVERY Chapter IV. Making Real Progress with the Requirements Defects Problem . 90 R. Geoff Dromey, Griffith University, Australia Chapter V. User Participation in the Quality Assurance of Requirements Specifications: An Evaluation of Traditional Models and Animated Systems Engineering Techniques 112 Heinz D. Knoell, University of Lueneburg, Germany Chapter VI. Academic Storytelling: The Development of User-Driven Scenarios 134 Robert Cox, University of Tasmania, Australia S ECTION III: P ROCESS C ONTRIBUTION TO IS Q UALITY Chapter VII. Process-Centered Contributions to Information Systems Quality 158 Evan W. Duggan, University of Alabama, USA Richard Gibson, American University, USA Chapter VIII. Developer-Driven Quality: Guidelines for Implementing Software Process Improvements 181 Gina C. Green, Baylor University, USA Rosann Webb Collins, University of South Florida, USA Alan R. Hevner, University of South Florida, USA Chapter IX. Improving Quality through the Use of Agile Methods in Systems Development: People and Values in the Quest for Quality 201 Julie E. Kendall, Rutgers University, USA Kenneth E. Kendall, Rutgers University, USA Sue Kong, Rutgers University, USA Chapter X. Quality Metrics and Bayesian Analysis: The Case of Extreme Programming 223 Francisco Macias, Technologic of Monterrey, Mexico Mike Holcombe, University of Sheffield, England S ECTION IV: M ANAGING R ISKS OF SPI P ROJECTS AND M ETHODOLOGIES Chapter XI. Building IT Risk Management Approaches: An Action Research Method 244 Jakob H. Iversen, University of Wisconsin Oshkosh, USA Lars Mathiassen, Georgia State University, USA Peter A. Nielsen, Aalborg University, Denmark Chapter XII. Examining the Quality of Evaluation Frameworks and Metamodeling Paradigms of Information Systems Development Methodologies 265 Eleni Berki, University of Tampere, Finland S ECTION V: IS Q UALITY I SSUES In U NDER -R ESEARCHED A REAS Chapter XIII. Software Quality and Open Source Process 291 Sameer Verma, San Francisco State University, USA Chapter XIV. Creating IS Quality in Government Settings . 311 Catherine Horiuchi, Seattle University, USA Chapter XV. ERP Quality: Do Individuals Matter? An Australian Case . 328 Jenine Beekhuyzen Griffith University, Australia About the Authors . 346 Index . 355 vi Foreword Evan W. Duggan (University of Alabama, USA) and Han Reichgelt (Georgia Southern University, USA) state the following in the first chapter of this impor- tant book on the quality of information systems: “Despite the fact that the IS discipline is over a half a century old and numerous articles have been written about software quality, there is, as yet, not a great deal of convergence of opinions on approaches for attaining quality.” Does this statement surprise you? Many global organizations have spent most of the past 20 years learning how to attain quality leadership in manufactured goods, the most basic of products. In the U.S., automotive companies are still playing catch up to Toyota and, most recently, Hyundai. Since the Japanese quality revolution in the late 1970s, to the most recent inva- sion of quality goods from Korea and China, we are now just learning what it takes to plan, control, and improve the quality of physical goods, let alone soft- ware or systems. Professors Duggan and Reichgelt have pulled together some of the latest re- search from capable authors to answer some of the most basic and important questions related to managing the quality of information and software systems. This book will become required reading for many including university and col- lege students becoming exposed to quality management techniques in their uni- versity classes; quality professionals who have been looking for answers to IT quality problems; and IT professionals that design products and services to meet the dynamic quality requirements of their customers. In today’s world, quality means “to attain the lowest level of failure, the highest level of quality at the lowest possible cost” as mandated by the customer. That is no small task in the IT world. In 1985, the world was just beginning to see the benefits of software develop- ment and IT systems. At that time, while working as an internal quality leader vii at the Perkin Elmer Corporation, a manufacturer of analytical instrumentation, I was asked, “how do we apply the quality management techniques that we learned from Juran, Deming, and others to non-manufacturing goods like soft- ware?” I was then a babe in the field of quality compared to quality gurus such as Dr. Joseph M. Juran (who was then 76 years old and is now 105!) but I had to figure out what to do. There were few books to read, few published cases on lessons learned, and even fewer “champions” that we could consult to answer the question: “how do we manage the quality of software and systems?” We did what we could and what we did was not much. It was not that important then because many citizens of this planet had not yet experienced the Internet, e-mail, or other IT innovations. Now it is 2006, and the scenario has not changed much for the quality of IT systems. We are still struggling to find answers to the question of how to man- age information system quality. Now as citizens of this planet, we rely on infor- mation systems that we did not care much about in 1985 to do just about every- thing. Information systems quality is now a “big thing” and “it needs improve- ment”. The IT revolution will continue into the coming years: Many organiza- tions are already seeing their internal IT infrastructures and people outsourced to larger more specialized companies; CIOs continue to change positions rap- idly; Moore’s Law will still be applicable; and organizations will continue to seek to improve the quality of IT systems to stay ahead of competition, or worse, avoid obsolescence. I am not a young lad anymore, nor am I as wise or as old as Dr. Juran but I have learned the secrets of attaining quality — good research, a high degree of col- laboration among experts, and a need to change. All three are here. The re- search conducted in preparation for this book has provided answers to some of the most common questions about IT and software quality. The collaboration on the part of these top researchers and practitioners in the field lend credibility to understanding the lessons learned. The need for change in IT and systems quality is here. Put all three together and business and society can reduce the pain of the sys- tem failures that plagued us in the last century. The Juran Institute has stated that we are moving into “the Century of Quality”. This century will be driven by the needs of society to have perfect quality. In the later part of the last century, we began to understand how to manage the quality of product. Now we have a better understanding of how to manage IT and software and systems quality. If we can figure out how to put the two (hardware and software quality) together, the citizens of this planet will enjoy this next century. Joseph A. De Feo , USA President & CEO Juran Institute Inc. viii Preface After years of experience with building systems, the information systems (IS) community is still challenged by systems delivery, that is, planning the imple- mentation project, determining system features and requirements, sourcing and deploying the software, and managing its evolution. High-quality systems are still elusive. Yet organizations invest heftily in IS to support their business op- erations and realize corporate priorities, either in search of competitive advan- tage or as a competitive necessity. One of the major paradoxes of our era is the disparity between the many innovations that have been enabled by information technology (IT) and the failure of the IS community, comprising developers, managers, and users, to exploit these advances to consistently produce high- quality IS that provide value to organizations. This phenomenon, which is high- lighted by Brynjolfssen (1993), Gibbs (1994), and others, has been dubbed “the software crisis.” The IS Quality Landscape The effects of the software crisis are demonstrated in the number of projects that are abandoned before completion (The Standish Group, 2003), deployed with poor quality, consuming inordinate maintenance resources (Banker et al., 1998), or remain unused after implementation (Markus & Keil, 1994). The IS community has rightly focused on how to reverse the trend of low-quality sys- tems, especially as IS become more central to the accomplishment of organiza- tional mission. The quality drive in IS delivery is reminiscent of the intensified focus on quality in the manufacturing and service areas, and the IS discipline has built on many of the concepts articulated by Deming and others. ix However, the systems delivery process has at least two key complicating fac- tors that are absent from most other manufacturing processes. First, IS are not confined to single operations, but typically address a network of interdependent business processes and interfaces to other technical systems (Liu, 2000), and are embedded within the social systems of the organization (Robey et al., 2001). Second, while the erection of physical structures takes increasing physical shape as the construction progresses, software artifacts remain “invisible” throughout development (Brooks, 1987). Progress is therefore much harder to monitor, and systems delivery more difficult to manage. It is little wonder that the pursuit of quality-enhancing approaches has taken many turns and has focused on a variety of factors. IS quality has been defined in various ways and encompasses several interpretations depending on the per- spective of the particular stakeholder. To address the many perspectives, the IS community has variously focused on the contribution of people, delivery pro- cesses, and development philosophies (methods) to the attainment of defined quality attributes of intermediary and final software products. User involvement in systems delivery facilitates the capture of the knowledge of appropriate domain experts, leads to greater commitment to system deci- sions and outcomes, and reduces the probability of usage failures (Barki & Hartwick, 1994). The IS community has also embraced the underlying logic of continuous process improvement programs, namely that process quality largely determines product quality (Deming, 1986). This has been substantiated in IS research (Harter et al., 1998, Khalifa & Verner, 2000; Ravichandran & Rai, 2000). Yet scholars have lamented that software process interventions are not well employed in IS organizations to improve quality (Fichman & Kemerer, 1997). Similarly, several systems delivery methods (design principles for driv- ing specific development techniques), such as rapid application development, object-oriented development, and agile methods (Duggan, 2004), all purport to improve IS quality. Despite the fact that the IS discipline is over a half a century old and numerous articles have been written about software quality, there is, as yet, very little convergence of opinions on approaches for attaining quality. In several cases, the same method attracts simultaneous claims of efficacy and ineffectiveness in different quarters. There is not even a commonly accepted definition for IS quality. This obviously affects agreement on important questions such as what are its determinants, what are the mechanisms by which quality is incorporated into the IS delivery process, and under what conditions are particular tech- niques likely to be successful. [...]... University SECTION I: INTRODUCTION AND OVERVIEW OF QUALITY CONCEPTS AND DIMENSIONS xxii The Panorama of Information Systems Quality 1 Chapter I The Panorama of Information Systems Quality Evan W Duggan, University of Alabama, USA Han Reichgelt, Georgia Southern University, USA Abstract Business organizations are still struggling to improve the quality of information systems (IS) after many research efforts and... Definition of key systems delivery terms Concept Information systems (IS) delivery Description All the activities involved in the overall planning, analysis, sourcing and deploying an information system The term delivery is used instead of “development” to account for all the possible methods (in-house development, purchase, outsourcing, rental, etc.) of sourcing an information system IS delivery paradigm... new systems and richer information flows to raise the quality of the design The chapter also explores the question of validation, which xiv is one of the primary issues inhibiting the wider use of storytelling or scenarios in information systems development The chapter demonstrates how the structured use of user-driven scenarios can assist practitioners in improving the quality of information systems. .. stages of the development process This has significant quality- enhancing implications Chapter V builds on the theme that improper specification of systems requirements has thwarted many splendid efforts to deliver high -quality information systems and links this problem largely to poor communication among systems developers and users at this stage of systems development It focuses on the devastating impact... permission of Idea Group Inc is prohibited The Panorama of Information Systems Quality 3 • 75 percent of all systems development undertaken is never completed or, if completed, remain unused (Lyytinen, 1988) • Runaway IS delivery projects — grossly over budget and hopelessly behind schedule — are prevalent (Mousinho, 1990) • The improvement of IS delivery quality is one of the top ten IS management problems... Section II: Quality in the Early Stages of IS Delivery Section II contains three chapters which examine various dimensions of IS quality, particularly in the early stages of the information systems life cycle System failures resulting from poor deliverables in the early stages of develop- xiii ment are pervasive (Dodd & Carr, 1994) It is generally acknowledged that quality, in the early stages of IS delivery, ... of the quality of the process of building software systems with agile development methods, which were designed to help with the development of higher quality information systems under given conditions The research assessed eXtreme Programming (XP), one of the several agile development approaches It compares XP with a traditional (design-driven) software construction process by observing and measuring. .. concept of scenarios (rich picture stories) to promote collaboration in the design of new information systems and provides insights into the opportunities available for the application of valid user scenarios Systems designers need contextual information about their users in order to design and provide information systems that will function effectively and efficiently within those contexts Storytelling... prescriptions for quality attainment We also recognize that there are other related, though not identical, concepts such as IS success which may or may not be determined by the objective quality of the system Copyright © 2006, Idea Group Inc Copying or distributing in print or electronic forms without written permission of Idea Group Inc is prohibited The Panorama of Information Systems Quality 9 The quality. .. correlated with IS quality; it may sometimes be partially — and occasionally wholly — determined by user perceptions In the ensuing discussion, we use this model as a basis for organizing the discussion of the contributions of people, process, and practices to product quality and further elaborate on the distinction we make in the model between IS quality and IS success Figure 1 Information systems quality model . Measuring Information Systems Delivery Quality Evan W. Duggan University of Alabama, USA Han Reichgelt. trademark. Library of Congress Cataloging-in-Publication Data Measuring information systems delivery quality / Evan W. Duggan and Han Reichgelt, editors. p.