High-Stakes Testing: Coping with Collateral Damage pptx

TeAM YYePG Digitally signed by TeAM YYePG DN: cn=TeAM YYePG, c=US, o=TeAM YYePG, ou=TeAM YYePG, email=yyepg@msn.com Reason: I attest to the accuracy and integrity of this document Date: 2005.07.06 15:19:12 +08'00' High-Stakes Testing: Coping with Collateral Damage This page intentionally left blank High-Stakes Testing: Coping with Collateral Damage R. Murray Thomas University of California, Santa Barbara NEA LAWRENCE ERLBAUM ASSOCIATES, PUBLISHERS 2005 Mahwah, New Jersey London Camera ready copy for this book was provided by the author. Copyright © 2005 by Lawrence Erlbaum Associates, Inc. All rights reserved. No part of this book may be reproduced in any form, by photostat, microform, retrieval system, or any other means, with- out prior written permission of the publisher. Lawrence Erlbaum Associates, Inc., Publishers 10 Industrial Avenue Mahwah, New Jersey 07430 www.erlbaum.com Cover design by Kathryn Houghtaling Lacey Library of Congress Cataloging-in-Publication Data Thomas, R. Murray (Robert Murray), 1921- High-stakes testing : coping with collateral damage / R. Murray Thomas. p. cm. Includes bibliographical references and index. ISBN 0-8058-5521-1 (cloth : alk. paper) ISBN 0-8058-5522-X (pbk.: alk. paper) 1. Educational tests and measurements—United States—Evaluation. 2. Education—Standards—United States. I. Title. LB3051.T467 2005 371.26'0973—dc22 2005040001 CIP Books published by Lawrence Erlbaum Associates are printed on acid- free paper, and their bindings are chosen for strength and durability. Printed in the United States of America 10 987654321 Contents Preface ix Chapter 1: The Nature of Collateral Damage 1 Examples of Success 2 Examples of Damage 5 The Book's Perspective 9 The Book's Structure 10 Learners' Characteristics 11 A Short History of Achievement Testing 12 Conclusion 20 Part I: Tasks of the Testing Game 23 Chapter 2: What Should the Schools Teach? 25 Deciding What the Schools Should Teach 25 Effect of Testing on What the Schools Do Teach 34 Conclusion 42 Chapter 3: What Should the Schools Evaluate and How? 45 A Host of Evaluation Methods 45 Test-Item Types—Strengths and Weaknesses 46 Achievement-Test Validity 50 Collateral Damage from High-Stakes Tests 51 Conclusion 56 Chapter 4: How Should Achievement Standards Be Set? 59 Alternative Ways to Set Standards 59 Standard Setting and Collateral Damage 66 Conclusion 75 vi High-Stakes Testing Chapter 5: How Should Test Results Be Used? 77 Six Traditional Functions of Test Results 78 Five No-Child-Left-Behind Functions 82 Grade Promotion and High-School Graduation 95 Four Frequently Affiliated Functions 99 Conclusion 105 Part II: The Testing Game's Players 107 Chapter 6: Politicians and Their Staffs 109 National-Level Politicians 110 State-Level Politicians 121 School-District Politicians 136 Conclusion 141 Chapter 7: Educational Administrators and Their Staffs 147 Nationwide Coping Strategies 148 State Coping Strategies 150 School-District Coping Strategies 160 Individual-School Coping Strategies 181 Conclusion 191 Chapter 8: The Public and Parents 193 Types of Coping Strategies 193 Conclusion 204 Chapter 9: Test-Makers and Test-Givers 205 Types of Coping Strategies 206 Conclusion 219 Chapter 10: Teachers 221 Types of Coping Strategies 221 Conclusion 239 Chapter 11: Students 241 Types of Coping Strategies 241 Conclusions 257 Chapter 12: Lessons to Learn 259 Lawmakers' Fitness for Making Decisions About Educational Reform 259 The One-Size-Fits-All Fallacy 263 The International Academic Olympics 264 Contents vii The Constraint of Finite Space 270 Static Model and Individualized Growth Model 272 The Value of Pilot Studies 283 Conclusion 284 References 287 Name Index 317 Subject Index 321 This page intentionally left blank Preface Early in the first decade of the 21st century, high-stakes achievement testing became the chief instrument that the U.S. federal government and state governments adopted for repairing what was deemed "the deplor- able condition of schooling in America." However, as testing programs were implemented, it became clear that achievement testing as the prime solution for students' unacceptable academic performance was accom- panied by a host of collateral damage in the form of vexing, unexpected, unwelcome problems. The four-fold purpose of this book is to (1) describe the nature of high- stakes testing, (2) identify types of collateral damage that have attended the testing programs, (3) analyze methods different groups of people have chosen for coping with the damage, and (4) suggest lessons to be learned from the high-stakes-testing experience. The six groups of peo- ple whose coping strategies are inspected include (a) politicians and their staffs, (b) educational administrators and their staffs, (c) parents and the public, (d) test-makers and test-givers, (e) teachers, and (f) students. ix [...]... Nature of Collateral Damage 5 accompanied by collateral damage The purpose of this book is to identify such damage and to analyze how people have tried to cope with it Examples of Damage The nature of collateral damage, its various causes, and ways of coping are described in detail throughout later chapters However, to gain an initial hint of the diverse sorts of damage that can be involved, consider the... initial examples of collateral damage, and (c) two kinds of background information useful for understanding the rest of the book Those two kinds are (a) significant characteristics of learners and (b) historical foundations of achievement testing At the start, it is important for readers to recognize the meanings assigned to the words in the title High-Stakes Testing Coping with Collateral Damage Testing... involved with testing cope with the damage The adjective high-stakes has been applied to the testing movement to suggest that the way the movement develops will yield serious consequences for students, school systems, and the society that schools are expected to serve The purpose of this opening chapter is to set the stage for subsequent chapters by introducing (a) some examples of the success of high-stakes. .. disgraceful ineffectiveness of the country's public schools However, as the high-stakes testing plan went into action, it became clear that its implementation was plagued by a distressing mass of collateral damage The dual purpose of this book is to identify multiple kinds of damage and to describe ways people have sought to cope with those kinds This page intentionally left blank Part I Tasks of the... chapter first describes alternative ways that the chapter's focal question can be answered and then identifies types of collateral damage that can result from how that question has been answered in typical high-stakes programs Each type of damage is not only described but is illustrated with one or more cases drawn mainly from newspaper accounts during the years following the 2002 introduction of the NoChild-Left-Behind... Set? Chapter 5: How Should Test Results Be Used? Part II, titled The Testing Game's Players and Their Coping Strategies, identifies the game's principal performers and analyzes their roles, their influence on the testing process, and their attempts to deal with the damage that so often accompanies high-stakes testing Chapter 6: Politicians and Their Staffs Chapter 7: Educational Administrators and... teaching students and evaluating their progress can lead to many of the kinds of 12 High-Stakes Testing collateral damage identified throughout this book In my experience, the most difficult task teachers face is that of accommodating the individual differences among the learners so that each student can progress in keeping with her or his own abilities, background of knowledge, and attitudes Clearly, tutoring... business leaders, and the general public were pleased with fourth-grade outcomes, which showed U.S pupils scoring above the international average (mean) of 500 in both math (545) and science (565)—higher than 19 The Nature of Collateral Damage 15 of the 26 nations participating in that portion of the study However, the U.S public was not so pleased with the eighth-grade results, which found American... nationwide achievement-testing program ever required for American public-school students That action, combined with states' current The Nature of Collateral Damage 21 achievement-testing schemes, would subject schools and their students to the most ambitious, costly, time-consuming, and complex high-stakes testing they had ever faced The purpose of this new testing movement was to correct what political... period, with the project annually affecting an estimated 117,000 students (Langland, 2003a) Under such circumstances, scores in those aspects of reading and math measured by the tests are bound to improve nationwide From that perspective, high-stakes testing has been, and will continue to be, a triumph However, such a victory comes at a price It requires trade-offs and is The Nature of Collateral Damage . 2005.07.06 15:19:12 +08'00' High-Stakes Testing: Coping with Collateral Damage This page intentionally left blank High-Stakes Testing: Coping with Collateral Damage R. Murray Thomas University . collateral damage. The purpose of this book is to iden- tify such damage and to analyze how people have tried to cope with it. Examples of Damage The nature of collateral damage, . identify types of collateral damage that have attended the testing programs, (3) analyze methods different groups of people have chosen for coping with the damage, and (4) suggest