Introduction
This chapter discusses the administration of the National Assessment of Educational Progress (NAEP), detailing student samples and achievement levels It offers a historical overview of the NAEP Reading Assessment Framework, highlights the development process of the 2009 NAEP Reading Framework, and compares it to earlier versions Additionally, it contextualizes the NAEP within the framework of the No Child Left Behind legislation.
Assessment Specifications
This chapter offers essential guidelines for the comprehensive development of assessments, detailing how to structure the entire item pool in alignment with the NAEP Reading Framework.
This chapter offers a comprehensive overview of text types used in assessments, outlines guidelines for selecting appropriate passages, and includes exhibits showing the distribution of passages across various grades and text types Additionally, it introduces the vocabulary assessment related to passage selection and discusses passage mapping, with further details provided in Appendix C.
General Item Specifications By Text Type, Cognitive Target,
This chapter outlines the item specifications relevant to all assessed grade levels, detailing the characteristics of NAEP Reading Assessment items It includes descriptions of various item formats and item-writing guidelines, as well as an introduction to the literary and informational text matrices Additionally, it presents a discussion on the cognitive target matrix, supported by relevant exhibits.
Inclusion of Students With Disabilities and English Language Learners
This chapter outlines essential accommodations and considerations for evaluating students with disabilities and English language learners It emphasizes the importance of item-writing strategies that are applicable to all assessment items, ensuring inclusivity and fairness in the evaluation process.
Vocabulary Assessment On the 2009 NAEP Reading Assessment
This chapter introduces a systematic approach to vocabulary assessment on the 2009 NAEP Reading Assessment It discusses the measurement of meaning vocabulary and includes criteria for selecting vocabulary to be assessed.
Scoring the NAEP Reading Assessment
This chapter discusses item types and score categories and provides guidelines for the development of scoring rubrics Alignment of items and scoring rubrics is also discussed.
Reviews and Item Tryouts
This chapter discusses the review and quality control procedures built into the assessment development process It focuses on item reviews including sensitivity reviews, classroom tryouts, and cognitive labs.
This chapter contains sample items illustrating the concepts and item-writing principles in Chapter 2 and the specifications for text types, cognitive targets, and grade levels described in Chapter 3
This chapter describes three recommended special studies that examine the assessment of vocabulary in context, achievement patterns of English language learners, and gender differences in reading achievement.
Special Studies
This chapter describes three recommended special studies that examine the assessment of vocabulary in context, achievement patterns of English language learners, and gender differences in reading achievement
Since 1969, the National Assessment of Educational Progress (NAEP) has served as a key national indicator of American students' knowledge and skills in major academic subjects, particularly in reading Regular assessments are conducted for students in grades 4, 8, and 12, with the No Child Left Behind Act of 2001 mandating biennial reading assessments for grades 4 and 8, and a quadrennial assessment for grade 12.
This Reading Assessment and Item Specifications for the 2009 National Assessment of
The Educational Progress document serves as a key resource for the 2009 NAEP Reading Assessment, targeting a technical audience such as the National Center for Education Statistics and the associated contractor It outlines essential details regarding passage selection, item development, and various facets of test development, effectively acting as the “test blueprint.”
The second document, the Reading Framework for the 2009 National Assessment of
Educational Progress, presents the conceptual base and content of the assessment and is intended for a more general audience
Since its inception in 1969, the National Assessment of Educational Progress (NAEP) has been a vital source of information on student achievement in reading, offering insights to citizens, curriculum specialists, and policymakers In 1988, Congress established the State NAEP program to assess reading and mathematics at the state level, utilizing the same instruments as the national assessments This program enables comparisons of student reading achievement across states and over time Additionally, the Long-Term Trend NAEP, administered in its original form since 1971, serves as the sole measure of long-term trends in reading achievement at the national level.
The National Assessment Governing Board (NAGB), established by Congress in 1988, is responsible for developing assessment objectives and specifications for the National Assessment of Educational Progress (NAEP) It sets achievement goals for different ages and grades while overseeing NAEP policy The NAEP Reading Assessment specifically measures reading comprehension in English, requiring students to read passages and answer related questions Notably, passages will not be read aloud to students, as the assessment focuses solely on reading comprehension While the assessment provides insights into student achievement, NAGB does not dictate how reading should be taught or endorse any specific curricular methods.
The reading passages included in the assessment are chosen for their appeal to students across the nation, ensuring they consist of high-quality literary and informational content while remaining unbiased Students engage with both multiple-choice and constructed-response questions The NAEP assessments for grades 4, 8, and 12 are comprehensive enough to provide valid results, although no single student completes the entire assessment; instead, each student reads about two passages and answers related questions.
NAEP assessments are conducted with random samples of students that represent the nation, various regions, participating states, and large urban districts As outlined in Chapter 3, NAEP results are provided for groups of students rather than individuals Since 1992, states have accessed data on students' reading achievement at the state level, while large urban school districts gained similar insights in 2002 and 2003 These results are published in documents like the NAEP reports.
Reading Highlights and the NAEP Reading Report Cards which are issued following each administration of the reading assessment; through special, focused reports; and through electronic means.
In 2002, the National Assessment Governing Board (NAGB) contracted the American Institutes for Research (AIR) to create the Framework and Specifications for the 2009 National Assessment of Educational Progress in Reading.
Three project committees were involved in making recommendations for the NAEP Reading Framework Development project:
A Steering Committee, consisting of representatives of the business community, national policy organizations, reading associations, and reading educators
A Planning Committee, consisting of reading educators in K–12 public and private education and in colleges and universities, business representatives, and members of education organizations
A Technical Advisory Panel, consisting of measurement experts from states, research organizations, and universities
Appendix A includes the members of the Steering and Planning Committees, as well as the Technical Advisory Panel Alongside these three project committees, the NAGB also established an External Review Panel made up of distinguished reading researchers and scholars to evaluate and offer an independent assessment of the draft 2009 NAEP Reading Framework.
Members of the External Review Panel are also listed in Appendix A.
2 National Assessment Governing Board (2002, May) National Assessment Governing Board policy on framework development Washington, DC: Author.
3 For more information about the development process and results, see the Reading Framework for the 2009
National Assessment of Educational Progress.
C ONTEXT FOR D EVELOPING THE A SSESSMENT AND I TEM
The 2009 NAEP Reading Framework and Reading Specifications were created during a period focused on enhancing reading achievement and informed by extensive research on reading development The No Child Left Behind (NCLB) Act of 2001 established clear objectives for the National Assessment of Educational Progress (NAEP), emphasizing the importance of assessing reading skills to ensure educational improvement.
The Progress Reauthorization Act (NAEPRA) of 2002 outlines the mission of the National Assessment of Educational Progress (NAEP), which is to deliver timely, fair, and accurate assessments of student academic achievement This includes reporting on trends in subjects such as reading and mathematics, among others.
The NAEP reading data will track trends in reading achievement at national, regional, state, district, and subgroup levels, without focusing on individual student or school performance In alignment with NCLB, the NAEP Reading Assessment will be conducted biennially for grades 4 and 8, with results reported promptly Additionally, while the public will have complete access to NAEP results and released test questions, NAEP will refrain from influencing the curriculum or assessments of any state The 2009 NAEP Reading Framework adheres to NCLB legislation.
In developing the 2009 NAEP Reading Framework, AIR, contracted by NAGB, undertook a thorough process that included an extensive review of scientific research, consultations with committees of national and state policymakers, state assessment staff, and reading educators, as well as a broad public review of multiple drafts of the Framework.
The 2009 NAEP Reading Assessment will be developed to represent the content emphasis, complexity of reading, item format guidelines, and other requirements of the NAEP Reading
Framework Item writers for the assessment will be experts in reading and reading education Under the direction of the test development contractor, they will use the Framework and these
Specifications to guide their work.
NAEP A DMINISTRATION AND S TUDENT S AMPLES
As currently planned, the 2009 NAEP Reading Assessment will assess students in grades 4,
The 2009 NAEP Reading Assessment for grades 8 and 12 will utilize three types of items: multiple-choice, short constructed-response, and extended constructed-response The assessment will feature multiple test booklets, employing a matrix sampling design to ensure that not all students receive the same items Alongside the reading items, background questionnaires will be included in separately timed sessions Each student is expected to spend approximately one hour completing the assessment.
The NAEP Reading Assessment measures reading achievement of students in the nation’s schools in grades 4, 8, and 12 and reports the results at national, regional, and state levels To
The National Assessment of Educational Progress (NAEP) Reauthorization Act mandates that schools across the United States are randomly selected to participate in assessments designed to reflect the diverse student population This sampling process ensures that schools of various types, sizes, and geographical locations are included, representing a wide range of economic statuses, racial, ethnic, and cultural backgrounds, as well as different instructional experiences Efforts are made to include students with disabilities and English language learners, with necessary accommodations provided to facilitate their participation.
The sophisticated matrix sampling strategy employed by the NAEP program enhances the generalizability of assessment findings across diverse student populations nationwide This approach enables the program to highlight aggregate student achievement strengths and weaknesses in reading, offer comparative data based on race/ethnicity, community type, and geographic region, track trends in student performance over time, and examine the relationships between student achievement and various background variables.
R EPORTING THE R ESULTS OF THE NAEP R EADING A SSESSMENT