Principles for Large-Scale Classroom-Based Teacher Assessment of English Learners’ Language An Initial Framework From School-Based Assessment in Hong Kong TRƯỜNG ĐẠI HỌC MỞ BÀI GIẢNG GIÁO TRÌNH

Greene, J A., & Azvedo, R (2007) A theoretical review of Winne and Hadwin’s model of self-regulated learning: New perspectives and directions Review of Educational Research, 77, 354–372 Leung, C (2007) Dynamic assessment: Assessment for or as teaching? Language Assessment Quarterly, 4, 257–278 Ramaprasad, A (1983) On the definition of feedback Behavioral Science, 28, 4–13 Smith, F., Hardman, F., Wall, K., & Mroz, M (2004) Interactive whole class teaching in the National Literacy and Numeracy strategies British Educational Research Journal, 30, 395–411 van Lier, L (1996) Interaction in the language curriculum: Awareness, autonomy and authenticity Harlow, England: Pearson Education Principles for Large-Scale Classroom-Based Teacher Assessment of English Learners’ Language: An Initial Framework From School-Based Assessment in Hong Kong LIZ HAMP-LYONS University of Nottingham Nottingham, England Ⅲ Davison and Leung (this issue) describe the field of teacher-based English language assessment as having “much variability, a lack of systematic principles and procedures and a dearth of information as to the impact of teacher-based assessments on learning and teaching” (p 389) In this Forum contribution, I briefly explore an example of teacher-based assessment of oral English that has been implemented across Hong Kong (see Davison 2007; Davison & Hamp-Lyons, in press) and the ways which elements of practice have contributed to or inhibited the establishment of the systematic key principles that are needed to establish any form of large-scale alternative assessment As this issue shows, assessment for learning is different from assessment of learning Black and Wiliam (1998) have summarized five elements that seem to be essential in the effectiveness of any attempt to improve learning by applying formative assessment or assessment for learning approaches: (a) the setting of clear goals; (b) the design of appropriate learning and assessment tasks; (c) the communication of assessment criteria to teachers and learners; (d) the provision of high quality feedback (both oral and written); and (e) the conscious provision of opportunities for self- and peer-assessment Assessment for learning gives a real role to the teacher as assessor of her/his own students’ learning 524 TESOL QUARTERLY and as judge of whether each student has met the expected standards of the subject at the level taught However, when assessment for learning meets formal examinations, paradigm conflict can emerge (Hamp-Lyons, 2007) In Hong Kong, the Hong Kong Examinations and Assessment Authority has implemented a policy of inserting a form of teacher-based assessment across most senior secondary subjects, locally referred to as school-based assessment (SBA), into the existing formal territory-wide examination system Although SBA looks very different in different subjects (see, e.g., Cheung, 2001, on teacher assessment in advanced-level physics, and Yung, 2001, on advanced-level biology), the ambitious aims of English SBA, of assessing authentic interactive use of the language in ways not easily assessed in formal examinations, have raised the possibility of a paradigm shift (Davison, 2007; Davison & Hamp-Lyons, in press).2 English SBA aims to bring together the highly formative aims of assessment for learning with the need to be meaningful at the level of the individual school and classroom, and to be accountable territory-wide Because it is carried out in every secondary fourth- and fifth-year English classroom in Hong Kong and the scores awarded by teachers comprise 15% of the English language public examination result, SBA still has to meet the traditional expectations of rigour for summative reporting, at the same time promoting formative assessment.3 Although research into and validation of SBA is ongoing, it has become clear that if this attempt to reconcile the formative goals of the innovation with system-wide accountability demands is to be successful, only a principled approach makes sense An expedient approach would be subject to the vagaries of shifting vested interests and the demands of different stakeholder groups Furthermore, interest in the innovation from other countries, especially in Asia, has already made it important to propose a statement of principles for SBA that might guide new adopters of this or a similar approach What follows is not—and cannot at this stage be—a fully developed theoretical and argued treatment of school-based language assessment principles, because of the wide range of test and assessment conditions into which a form of SBA might be introduced Rather, it is an attempt to extrapolate some early-stage principles for further The basic plan for an interactive, reading-based, assessment of students’ oral English proficiency by classroom teachers grew from a major curriculum and assessment reform (IBM, 2003, and the Consultancy Team of Chris Davison and Liz Hamp-Lyons, with a team of research staff at The University of Hong Kong, have since early 2005 worked on the research, design, development, standards setting, and implementation of the new assessment, funded by the Hong Kong Examinations and Assessment Authority, and also on several research and teacher capacity-building projects funded by the Quality Education Fund and the Hong Kong Research Grants Council SBA results play a significant part in deciding whether or not a young person continues schooling and aims for university or leaves education at this stage THE FORUM 525 argumentation and validation from the development decisions made during the four-plus years of work on introducing into the Hong Kong Certificate of Education (English) exam a classroom-based, teacher-led speaking assessment It is an attempt to formulate a more specifically-targeted set of principles for SBA than has previously been available in the educational measurement literature The principles proposed in this brief article also acknowledge the specific nature of language assessment in that, with the assessment of oral interaction skills based on a reading–viewing programme of input, we are simultaneously assessing medium and message The work of Marion and Pellegrino (2006) has been of particular value Marion and Pellegrino argue that, for alternative assessment, a new model of assessment is required that accounts for the relationship between the characteristics of the students tested and how they develop “domain competence” (p 52), the nature of observation of student performance, and the ways in which observation data are to be interpreted Marion and Pellegrino also stress that “decision consistency” (p 52) is a key quality to be achieved by alternative assessments within a standardsreferenced reporting framework (as is the case in Hong Kong), a more humanistic view of what mainstream educational measurement would refer to as reliability However, the rapidly growing international assessment-for-learning movement holds as a principle that a classroom-based, teacher-led assessment that is intended to directly affect the quality of teaching and learning must ensure that the teachers themselves are enabled to conduct valid assessments In the case of school-based, teacher-conducted assessment, teachers will require extensive professional development For Hong Kong English oral SBA, a comprehensive set of DVDs was developed, comprising a 24-hour programme of professional development (PD) which can be delivered by trained teacher educators (SBA Consultancy Team, 2006) These PD materials have emphasized the close connections between teaching, learning, and assessment, and teachers have been shown models of lesson sequences, sample texts and sources, model tasks, relevant readings from the professional literature as well as short input texts on key principles written by the consultants, and an array of student performances at a wide range of levels These PD courses have been delivered to more than 4,000 teachers to date The key principles of a test or assessment instrument are commonly referred to as reliability, validity, practicality, and (sometimes) consequences However, other formulations are emerging, including McMillan’s (2000), which proposes principles that embody “the big ideas that, when effectively applied, will effectively guide good assessment practices, regardless of the grade level, subject matter, developer, or user of the results” (p 1) McMillan argues that 526 TESOL QUARTERLY Teacher assessment is a process of professional judgement requiring understanding of basic measurement principles and a critical evaluative ability to effectively interpret evidence from student performances and behaviour and make value judgments about the meaning of results Assessment decisions are the result of a series of tensions or trade-offs (e.g., between formative and summative, or between criterion referenced and norm referenced) Assessment influences student motivation and learning.4 Good assessment must be aware of both its intended and unintended consequences Good assessment is fair Good assessment uses multiple methods and sources of evidence Assessment must be efficient and feasible as teachers and schools have limited time and resources.5 These principles are valuable, but in developing and implementing SBA, the Hong Kong research team felt the need to elaborate on particular areas that are less commonly discussed and are not well understood in Hong Kong and many other Asian contexts These elaborations are emerging not only from the educational measurement literature but from empirical interview data collected in a 4-year longitudinal study of the SBA implementation: A classroom-based assessment should foster student learning and development by (a) ensuring that all students have equitable access to the resources, tools, and information they need to succeed; (b) providing tasks that are responsive to the needs and abilities of students within the school and the class and that stimulate the authentic use of English to read and view texts, and to talk about texts; (c) providing equitable opportunities for students to learn the relevant materials, skills, and knowledge to succeed in the curriculum; (d) building the capabilities and motivation of teachers to improve students’ teaching and learning through appropriate, timely, and accessible professional development McMillan cites Wiggins (1998), who uses the term educative assessment to describe techniques and issues that educators should consider when they design and use assessments in order to ensure that the assessments themselves lead students to have meaningful engagement in the learning process The greatest resistance to SBA has come from teachers who perceive it to be a major added burden in terms of time and workload, which suggests that further teacher professional development is needed along with even more support from education authorities THE FORUM 527 Validity for a classroom-based assessment must take special account of the construct being assessed by (a) defining the domain to be assessed (in the case of SBA, speaking proficiency) and ensuring an appropriate and fully representative form of observation of student performance (construct representativeness); (b) accounting for the relationship between the characteristics of the students tested (starting point, needs, opportunities to learn, etc.) and the pathways by which they develop competence in the ability to be assessed (construct relevance); (c) systematically documenting how the observation data are to be interpreted (accountability) In the specific case of school-based assessment, it is particularly important for teachers and administrators to understand their roles in gathering and interpreting validity evidence A classroom-based assessment should foster an atmosphere of fairness in teaching, learning, and assessment by (a) including equal opportunity to learn as a key aspect of fairness in classroom-based assessment; (b) ensuring that scores reported for individual students are based on the best possible sample of language performance that can be obtained within the classroom context; (c) ensuring that teachers within each school meet regularly to discuss student performances within the year but between classes; (d) ensuring that teachers from different schools can meet occasionally to view and discuss students’ performances, reminding them of the wide variation in student abilities across the whole school system; (e) providing professional development opportunities to teachers to improve their ability to see the links between observing performances; recording strengths, weakness, and needs from those observations; and building support and/or further challenges for learners as appropriate; (f) recognizing that fairness is as much a sociocultural as a technical construct.6 A classroom-based assessment should foster educational improvement and accountability through a system-wide process by (a) providing evidence of student and school performance using multiple indicators of student learning over sufficient time to demonstrate improvement; 528 Different cultures vary in their interpretation of fairness; it is not just a technical issue This aspect of the SBA implementation is discussed in an article by the author currently under review TESOL QUARTERLY (b) (c) ensuring that teachers and involved administrative staff are provided with training and opportunities to improve in areas where they find themselves to be weak; carrying out ongoing quality assurance and validation The final issue, or principle, that needs to be considered in shepherding any assessment innovation toward success is the powerful role played in education by politics within an education system SBA, and any new approach to assessment, cannot be expected to be successful in the classroom unless conditions at higher levels—the school, the education system, the state, and the society as a whole—are sympathetic to reform Thus, the need for educational and social activism is one that cannot be ignored THE AUTHOR Liz Hamp–Lyons is Special Professor in English Language Education at the University of Nottingham, England Her research interests include the development and validation of English language performance assessments; rater training; English language teacher professional development, especially in school-based assessment; writing pedagogy, theory, and materials for teaching writing; language programme evaluation; advanced academic literacy; and English for academic purposes REFERENCES Black, P., & Wiliam, D (1998) Assessment and classroom learning Assessment in Education, 5, 7–75 Cheung, D (2001) School-based assessment in public examinations: Identifying the concerns of teachers Education Journal, 29, 109–123 Davison, C (2007) Views from the chalkface: School-based assessment in Hong Kong Language Assessment Quarterly, 4, 1–32 Davison, C & Hamp-Lyons, L (in press) The Hong Kong Certificate of Education: School-based assessment reform in Hong Kong English language education In L.-Y Cheng & A Curtis (Eds.), English language assessment and the Chinese learner New York: Routledge Hamp-Lyons, L (2007) The impact of testing practices on teaching: Ideologies and alternatives In J Cummins & C Davison (Eds.), The international handbook of English language teaching, (Vol 1, pp 487–504) Norwell, MA: Springer IBM (2003) Strategic review of the public examination system: Final consultancy report Hong Kong SAR, China: Examinations Authority Marion, S F., & Pellegrino, J W (2006) A validity framework for evaluating the technical quality of alternative assessments Educational Measurement: Issues and Practice, 20(4), 47–57 McMillan, J H (2000) Fundamental assessment principles for teachers and school administrators Practical Assessment, Research & Evaluation, 7(8) Retrieved March 14, 2007, from http://PAREonline.net/getvn THE FORUM 529 SBA Consultancy Team, Faculty of Education, University of Hong Kong (2006) Professional development courses for teachers in preparation for the school-based assessment in HKCE English language examination [visit http://web.hku.hk/~sbapro/research html for further information] Wiggins, G (1998) Educative assessment: Designing assessments to inform and improve student performance San Francisco: Jossey-Bass Yung, B H W (2001) Examiner, policeman or students’ companion: Teachers’ perceptions of their role in an assessment reform Educational Review, 53, 251–260 Issues in the Development of a Descriptor Framework for Classroom-Based Teacher Assessment of English as an Additional Language CATRIONA SCOTT Newcastle University Newcastle upon Tyne, England Ⅲ This article identifies some key conceptual issues in teacher-based assessment arising from the development of a framework for formative classroom-based teacher assessment of learners with English as an additional language7 (EAL) in publicly funded schools in England.8 Teacher-based assessment, particularly for formative purposes, has become increasingly prominent in educational policy documents both in the United Kingdom and internationally Building on Black and Wiliam’s (1998a, 1998b) early work on formative assessment, there is increasing evidence that such assessment, also referred to as assessment for learning (AfL) (see, e.g., Black et al., 2003), can be a powerful tool in promoting learning In England there is no separate framework for the assessment of EAL learners, who are assessed according to National Curriculum Levels, English as an additional language (EAL) and English as a second language (ESL) both refer to “learners who are using English as the medium of instruction in school contexts but who are not English first language (L1) speakers” (Rea-Dickins 2000, p 115), as distinct from English as a foreign language (EFL) contexts in which English is not the lingua franca Where these terms occur in this article, they are used synonymously This contribution draws on the EAL Assessment Frameworks Research Project jointly funded by the National Association for Language Development in the Curriculum (NALDIC) and the Paul Hamlyn Foundation (2002–2003), which surveyed six assessment EAL/ESL frameworks from Australia, the United States, Canada, and the United Kingdom, and also my recent experience in developing a draft formative assessment framework for trialing in elementary schools 530 TESOL QUARTERLY