Essential readings on assessment

Essential Readings ON Assessment Compiled and introduced by Peter Afflerbach IRA BOARD OF DIRECTORS Patricia A Edwards, Michigan State University, East Lansing, Michigan, President • Victoria J Risko, Peabody College of Vanderbilt University, Nashville, Tennessee, President-elect • Carrice C Cummins, Louisiana Tech University, Ruston, Louisiana, Vice President • Janice F Almasi, University of Kentucky, Lexington, Kentucky • Rizalina C Labanda, Sts Peter and Paul Early Childhood Center, Laguna, Philippines • Marsha M Lewis, Duplin County Schools, Kenansville, North Carolina • Karen Bromley, Binghamton University, SUNY, Binghamton, New York • Brenda J Overturf, University of Louisville, Louisville, Kentucky • Terrell A Young, Washington State University, Richland, Washington • Jay S Blanchard, Arizona State University, Tempe, Arizona • Kathy Headley, Clemson University, Clemson, South Carolina • Joyce G Hinman, Bismarck Public Schools, Bismarck, North Dakota The International Reading Association attempts, through its publications, to provide a forum for a wide spectrum of opinions on reading This policy permits divergent viewpoints without implying the endorsement of the Association Executive Editor, Books    Corinne M Mooney Developmental Editor    Charlene M Nichols Developmental Editor    Stacey L Reid Editorial Production Manager    Shannon T Fortner Design and Composition Manager    Anette Schuetz Project Editors    Stacey Reid and Wendy L Logan Cover Design    Linda Steere Copyright 2010 by the International Reading Association, Inc All rights reserved No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, or any information storage and retrieval system, without permission from the publisher The publisher would appreciate notification where errors occur so that they may be corrected in ­subsequent printings and/ or editions Library of Congress Cataloging-in-Publication Data Afflerbach, Peter Essential readings on assessment / Peter Afflerbach p cm Includes bibliographical references ISBN 978-0-87207-812-3 Reading Ability testing Reading (Secondary) Reading (Middle school) I Title LB1050.46.A35 2010 428.4’076 dc22 2010028953 Suggested APA Reference Afflerbach, P (Ed.) (2010) Essential readings on assessment Newark, DE: International Reading Association Contents About the Editor      v Introduction      Peter Afflerbach Teachers as Evaluation Experts      13 Peter Johnston Reading Assessment: Time for a Change      18 Sheila W Valencia and P David Pearson Meeting AYP in a High-Need School: A Formative Experiment      24 Douglas Fisher, Nancy Frey, and Diane Lapp Developing the IRIS: Toward Situated and Valid Assessment Measures in Collaborative Professional Development and School Reform in Literacy      35 Theresa Rogers, Kari Lynn Winters, Gregory Bryan, John Price, Frank McCormick, Liisa House, Dianna Mezzarobba, and Carollyne Sinclaire A Framework for Authentic Literacy Assessment      46 Scott G Paris, Robert C Calfee, Nikola Filby, Elfrieda H Hiebert, P David Pearson, Sheila W Valencia, and Kenneth P Wolf Reading Fluency Assessment and Instruction: What, Why, and How?      58 Roxanne F Hudson, Holly B Lane, and Paige C Pullen QAR: Enhancing Comprehension and Test Taking Across Grades and Content Areas      73 Taffy E Raphael and Kathryn H Au Tile Test: A Hands-On Approach for Assessing Phonics in the Early Grades      89 Kimberly A Norman and Robert C Calfee Assessing Adolescents’ Motivation to Read      101 Sharon M Pitcher, Lettie K Albright, Carol J DeLaney, Nancy T Walker, Krishna Seunarinesingh, Stephen Mogge, Kathy N Headley, Victoria Gentry Ridgeway, Sharon Peck, Rebecca Hunt, and Pamela J Dunston Teacher Questioning as Assessment      119 Peter Afflerbach Focused Anecdotal Records Assessment: A Tool for Standards-Based, Authentic Assessment      135 Paul Boyd-Batstone “I’m Not Stupid”: How Assessment Drives (In)Appropriate Reading Instruction      146 Danielle V Dennis Parents and Children Reading and Reflecting Together: The Possibilities of Family Retrospective Miscue Analysis      155 Bobbie Kabuto Assessing English-Language Learners in Mainstream Classrooms      166 Susan Davis Lenski, Fabiola Ehlers-Zavala, Mayra C Daniel, and Xiaoqin Sun-Irminger Show Me: Principles for Assessing Students’ Visual Literacy      178 Jon Callow Using Electronic Portfolios to Make Learning Public      190 Kevin Fahey, Joshua Lawrence, and Jeanne Paratore About the Editor Peter Afflerbach is professor of reading education in the Department of Curriculum and Instruction at the University of Mary­ land, College Park, USA He teaches graduate and undergraduate courses in reading assessment, models of reading, and reading comprehension Peter received his master’s degree in reading education and PhD in reading psychology from the University at Albany, State University of New York, USA Prior to becoming a professor he taught grade K–6 Title reading, middle school remedial reading and writing, and high school English in public schools in New York Peter has authored numerous books, most recently Understanding and Using Reading Assessment, K–12 (2007) and Adolescent Literacy Inventory, Grades 6–12 (with William Brozo, 2010) His work appears in numerous theoretical and practical journals, including Reading Research Quarterly, Cognition and Instruction, The Elementary School Journal, Journal of Reading, Journal of Reading Behavior, The Reading Teacher, Journal of Adolescent & Adult Literacy, and Language Arts He is also the author of High Stakes Testing and Reading Assessment, a National Reading Conference policy brief Peter is a coeditor of the Handbook of Reading Research (4th edition, 2011), the Assess­ ment section of the Handbook of Research on Teaching the English Language Arts (2011), and the research journal Metacognition and Learning He has served on National Assessment of Educational Progress committees, including the 2009 Reading Framework Committee and the standing Reading Committee, and on review panels for the Common Core Standards for English Language Arts & Literacy in History/Social Studies, Science, and Technical Subjects Peter chaired the International Reading Association’s Issues in Literacy Assessment Committee and served on the IRA–NCTE Joint Task Force on Assessment, which recently produced the Standards for the Assessment of Reading and Writing (2010) Peter was elected to the Reading Hall of Fame in 2009 v Introduction Peter Afflerbach R text, to effectively identifying details and main ideas, to employing prior knowledge to construct meaning, to critically appraising meaning Most recently, the conceptualization of reading includes comprehending text and using the constructed meaning of texts, including multimedia texts and those found on the Internet, to conduct related tasks As conceptions of reading change so, too, must reading assessment For example, when the ability to provide verbatim recall was equated with accomplished reading, an effective assessment involved matching the reader’s spoken or written recall with the actual text and determining the degree of match When the epitome of good reading was considered the ability to identify details and main ideas, multiple-choice questions that required readers’ correct selection of main idea and details from competing choices was suitable If, however, the good reader is one who constructs meaning from texts and then uses the constructed meaning in a task, then assessment should describe the reader’s achievement in related terms For our developing student readers, synthesizing information from different texts, combining it with graphic and pictorial information, then using this information to help solve problems could serve as suitable assessment Effective assessment demonstrates alignment of four levels: (1) the construct of reading, (2) reading standards and benchmarks, (3) reading curriculum and instruction, and (4) reading assessment Understandings from research and practice inform our conceptualization of reading and reading development They describe the construct of reading, from which district, state, and national reading standards and benchmarks are derived Curriculum and instruction follow; mapped carefully onto standards and benchmarks, curriculum eading assessment informs our understanding of individual students and our related efforts to best teach them, which shapes students’ self-concepts, motivations, and attitudes related to reading It signifies, to many, the success and accountability of our teachers and schools To serve these important and varied purposes, assessment should be involved in our daily classroom routines, and it should measure students’ and teachers’ long-term accomplishments Reading assessment should tell the story of developing readers’ learning and mastery of the mechanics of reading and of their ability to construct literal, inferential, and critical understandings The assessment should also help us understand if, and how, students are developing enthusiasm for reading It should report on, and support, student reading development Given the importance of assessment in the lives of readers, this volume offers a collection of readings taken from the archives of the International Reading Association (IRA) These readings focus on different, critical aspects of effective reading assessment The Current State of Reading Assessment What is reading? How we conceptualize it? How we foster our students’ reading development? Our experiences, and the detailed research base on reading and literacy, best inform the answers to these questions Over the centuries, ideas about how reading is conceptualized and what qualifies a reader as accomplished have undergone continual revision (Kaestle, 1991) The mark of the good reader has evolved from providing verbatim recall of a recently read Essential Readings on Assessment, edited by Peter Afflerbach © 2010 by the International Reading Association and instruction are the means for helping students approach and meet the goals Reading assessment, derived from curriculum and instruction, reports on the achievement of student readers in relation to curriculum and instruction Each of the four levels must be aligned, and the direct connection between the construct of reading and reading assessment serves as a test of the construct validity of assessment Today our progress in understanding how reading works, and how developing readers grow, is not fully reflected in many current reading assessments The reasons for this are varied, but they combine to present the ongoing challenge of making sure that how we think about reading and how we think about reading assessment are informed by our most recent and comprehensive understandings of both Understanding Reading We should create and use reading assessment that best reflects our current and robust understandings of what reading is, how it works, and how it is used We should also be wary of reading assessments and programs that provide only a partial picture of our students’ reading challenges and accomplishments Effective assessment helps us understand both developing readers and established readers Thus, our assessment programs should focus on the core skills and strategies that developing readers must learn and master, as well as the increasingly complex thinking that we expect of students For example, effective assessment helps us understand the developing reader’s phonemic awareness and use of sound–symbol correspondence knowledge Such assessment is relatively straightforward— we can determine students’ phonics knowledge through examination of their writing, miscue analysis of their oral reading, and matching exercises Fluency may be examined as students read aloud and we determine rate, accuracy, and prosody (Rasinski, 2006) This information helps us understand how readers establish competence in the mechanics of reading Reading assessment currently focuses, almost exclusively, on the reading skills and Afflerbach strategies highlighted by the National Institute of Child Health and Human Development (2000) and related No Child Left Behind (NCLB) legislation These are the “five pillars” of reading: (1) phonemic awareness, (2) phonics, (3) fluency, (4) vocabulary, and (5) comprehension These important skills and strategies are but a part of the construct (and related standards and instruction) of reading Yet, more recent efforts represent the evolving understanding of reading and related curriculum and instruction Consider the characterization of reading provided in the Reading Framework for the 2009 National Assessment of Educational Progress (National Assessment Governing Board, 2008): Reading is an active and complex process that involves: • Understanding written text • Developing and interpreting meaning • Using meaning as appropriate to type of text, purpose, and situation (p iv) This definition represents reading as both constructing meaning and using that constructed meaning Thus, assessment should help us understand how students construct meaning from text, the nature of the constructed meaning, and how they use the constructed meaning in related tasks Assessment of a student’s ability to determine a main idea or to make inferences from text remains important information, but it cannot tell the full story of how a student is growing as a reader T he Com mon Core St at e St a nd a rds (Common Core Standards Initiative, 2010) propose reading benchmark performances that also push the boundary of how reading is conceptualized Consider, for example, the following 12thgrade reading standard for informational text: It is expected that students meeting standard will “integrate and evaluate multiple sources of information presented in different media or formats (e.g., visually, quantitatively) as well as in words in order to address a question or solve a problem” (p 40) This standard reflects our understanding of the strategies and mindsets that are needed when a reader reads two or more texts on a related topic In such reading, the reader must create single understandings of each text, then combine these understandings to create a constructed meaning that accommodates what may be conflicting information in the different texts The preceding portrayals of reading assume that readers have learned and mastered prerequisite skills and strategies and that they possess appropriate prior knowledge and stances toward reading to undertake such reading These developmental reading achievements are an important focus for reading assessment In fact, they receive the bulk of attention as students progress through elementary school Thus, a challenge is to judiciously plan the focus of teaching and assessment, making sure that reading in school fully represents the construct of reading and that reading assessment is best situated to measure and describe reading Understanding Assessment As our understanding of reading evolves so, too, does our knowledge about effective assessment Pellegrino, Chudowsky, and Glaser (2001) propose a model of assessment that includes cognition, observation, and interpretation Cognition—the important cognitive skills and strategies that are necessary for successful reading and the content that is learned from reading in school—is the primary focus of reading assessment Observation includes the specific assessment materials and procedures that are used to gather assessment information Interpretation involves the inferences that we make from our assessment data When we have confidence in the alignment of our assessment (observation) with the thing(s) to be measured (cognition), we can make appropriate inferences (interpretation) about student reading achievement (i.e., what a student has accomplished and what remains to be learned) I have several observations related to Pellegrino et al.’s (2001) model of assessment First, it focuses exclusively on the cognition component of reading—it applies to the reading skills and strategies that readers develop, as well as the knowledge that is constructed about content through acts of reading Second, and relatedly, the model does not focus on reader motivation or affect, components of reading that are known to influence both individual acts of reading and lifelong reading behaviors Third, the model is scalable: It speaks to the important acts of assessing phonemic awareness, as well as fluency and reading comprehension Last, the model provides strong argument for how valid, effective reading assessment emanates from state-of-the-art knowledge of reading and assessment My colleagues and I (Afflerbach, 2007; Leipzig & Afflerbach, 2000) proposed the CURRV model, which focuses on the consequences, usefulness, and roles and responsibilities related to the assessment, as well as the traditional benchmarks of reliability and validity, when we plan our reading assessments For example, if a school district contemplates adopting a performance assessment program, it must focus on consequences that include the performance assessment’s ability to inform instruction, provide ongoing formative assessment information, and help teach students to use rubrics and approach independence in self-assessment As well, performance assessments demand considerable teacher and student expertise, and how teachers and students are supported as they work toward this expertise in performance assessment is a key consideration The a priori consideration of any and all consequences of assessment puts us in better position to make judgments and decisions to adopt, support, or question a particular type of assessment A consequential validity perspective proposes that we consider the consequences of an assessment, or of the use of assessment results, as part of an assessment development or selection process Are the labels above average, average, and below average appropriate? Whatever an assessment tells us about a student’s cognitive skills and strategies and learning of text content, does it make sense to influence students’ self-esteem and self-concept as a reader with such words? What are the consequences of being labeled a below-average reader throughout one’s school career? Words matter (Johnston, 2004), and our consideration of the consequences of the language we use to describe students could not be more important Introduction