Studies in Immigrant English Language Assessment. Volume 2.

Studies in Immigrant English Language Assessment Volume Geoff Brindley and Catherine Burrows (Eds) Sydney, Australia: National Centre for English Language Teaching and Research, Macquarie University, 2001 Pp viii + 187 Ⅲ In the last years, the Australian Adult Migrant Education Program (AMEP) has conducted a number of projects investigating the current state of ESL immigrants, the results of which have been presented in the two-volume Studies in Immigrant English language Assessment This series of books intends to provide current experiences, quantitative and qualitative data, and research indicators in order to improve the reliability of assessment in the Certificate in Spoken and Written English (CSWE) In this sense, Volume brings very interesting and valuable information to fill the current literature gap in initial language assessment for immigrants In the introduction, Burrows gives detailed information about the AMEP programme, its background and history, and about the CSWE, REVIEWS 353 which is a competency-based assessment tool to measure the immigrants’ language and literacy skills The test includes four certificate levels: speaking, listening, reading, and writing She also addresses different issues related to the CSWE, for example, the role and description of assessment, current research, studies, and projects related to the certificate The book’s main body covers “a range of assessment concerns, each focusing, in a different way, on classroom-based assessment” (p 7) In chapter 1, Claire examines moderation procedures and the rating system used for CSWE assessment through a survey given to 43 AMEP teachers with questions about assessment methods, the assessment of oral competence, and the use of audio/videotaping as recording tools The survey concludes that (a) raters range from formal to informal assessment according to the difficulty of the students’ assignment and the final written performance, and (b) audio- and videotaping are valuable tools that affect the quality of CSWE assessment In the qualitative analysis of teachers’ responses to the students’ benchmark1 texts, Claire uses the moderation of written competence and its videorecording She finds significant differences in the performance criteria, the moderation session format, and the relationship between the number of texts assessed, each student’s measured capability, and general rating results Thus, she concludes that more resources are needed when conducting moderation exercises to guarantee a better delivery of the certificate (namely staff and recording equipment), and that there should be a focus on the student’s typology to better address his or her needs In chapter 2, Brindley reports on a quantitative study of oral and writing elements of competence assessment (a total of 22 spoken videotaped and 72 written texts from several AMEP providers) and consistency patterns in the classification practices as practiced by (only) 12 experienced CSWE assessors Among Brindley’s significant observations was the issue of the raters’ irregularity of assessment (as measured by an ANOVA) Despite continuous training and feedback, significant discrepancies in grading and application of individual performance criteria for the same exercise (among individual raters and when hired raters are compared with the CSWE providers) are observed in two areas: performance interpretation criteria and criteria in assessing not previously established assessment elements Brindley recommends paying special attention to these two areas when conducting their rating procedures, so as to avoid discrepancies among correctors According to Brindley, consistency can also be improved by compiling banks of texts to be used as examples for each certificate, to provide guidelines to identify their key 354 Defined as samples “which illustrate a minimum standard of achievement of a competency” (p 60) TESOL QUARTERLY features, to eliminate rate criteria which lead to divergent interpretations, and to provide more professional development The chapter also includes a text suitability checklist (p 81) as well as the CSWE reading assessment tasks used in moderation sessions (p 85) The second part of this volume addresses the washback effect in CSWE assessment (the effects of the introduction and use of a test on language teachers and learners) After giving a brief historical background of the CSWE assessment procedures, Burrows (chapter 3) looks at how teachers may introduce a new CSWE assessment system into their class work through surveys, interviews, and classroom observations The survey results showed little washback effect overall In her survey, she finds that some teachers show a certain degree of resistance to the implementation of the assessment system because of their “unconviction of the benefits of the implementation of the assessment” (p 165) She also gives a taxonomy of the changes observed since the implementation of the new guidelines and practices (materials, methods, and teacher’s and students’ behaviour) On the whole, the survey phase remained inconclusive about washback effects Burrows concludes with classroom observations in which she followed the classification of teachers according to their attitude toward changes in teaching due to the washback effect She distinguishes between three groups (based on practices of resistance and adaptability to innovations at the level of instruction) The types reflect attitudes towards washback and change: adopters (when the new teacher adds the new assessment elements to his or her classes with little or no significant modifications), adapters or partial adopters (who try to balance current practice with innovations), and resisters (who are reluctant to accept, and in some cases even reject, the new ideas) Like other researchers, Burrows stresses the fact that if changes or modifications in practices are to be implemented, it is the teacher who has the final say By appealing to their professionalism, changes and improvements in the current instructional procedures are possible The chapter concludes with an annotated bibliography of 153 items The volume does have some shortcomings, however Although the back cover copy notes that the book will interest “teachers, curriculum developers, researchers,” many readers may soon be disappointed when they come across statistical analysis and conclusions that seem to support the researchers’ own ideas but are empirically inconclusive For instance, in chapter the sample is very small and also the subjects are unrepresentatively experienced and highly qualified raters Also, the sample selection analysed in chapter seems very limited The conclusions provided in chapters and not seem universally reliable nor valid even for the AMEP programme The volume also lacks a good conclusive chapter to summarize the outcomes of the reports in the three chapters REVIEWS 355 On the whole, however, this book is a valuable review of the many different realities behind the CSWE certificate Readers in the field of large-stakes testing will find it worth reading and will find a variety of real testing materials used to exemplify the key points as well as some 40 clear and well-thought-out tables and figures It is a highly regarded first academic approach to the difficult issue of initial language assessment for immigrants JESÚS GARCÍA LABORDA Universidad Politécnica de Valencia Valencia, Spain 356 TESOL QUARTERLY