Current issues in english language teacher based assessment

DEFINING TBA There is no widely accepted common defi nition of teacher-based assess-ment in the English language teaching fi eld, with many terms used interchangeably to refer to the

Current Issues in English Language

on traditional, but now outdated, psychometric assumptions This cle provides an overview of some of the current issues in TBA, includ-ing its defi nition and key characteristics, and the complex but signifi cant questions which its implementation pose for our understandings of lan-guage, learning, and assessment

Teacher-based assessment (TBA) is policy-supported practice in a number of educational systems internationally, including Australia, New Zealand, 1 Canada, and the United Kingdom (e.g., Cumming & Maxwell, 2004; Learning and Teaching Scotland, 2006 2 ; Queensland Studies Authority, 2009b; Saskatchewan Learning, 1993; Spencer, 2005)

It is increasingly being adopted as national educational policy in Asia

1 In Queensland where school-based assessment (SBA) was introduced in the 1970s (Sadler, 1989) teacher-based assessment is used for all assessment in the secondary school, even for high-stakes purposes (see Queensland Studies Authority, 2009a) The Australian Capital Territory (ACT) also uses only teacher-based assessment for senior secondary level (Department of Education and Training, Education Policy and Planning Section [Australia], n.d.) Other states such as New South Wales and Victoria have incorporated large scale teacher-based assessment into their public examinations (see, e.g., New South Wales Government, n.d.) New Zealand also has a long history of school-based assessment

in the senior secondary school (see New Zealand Qualifi cations Authority, n.d.), and has developed a wide variety of teacher support material and associated research studies, (Ministry of Education [New Zealand], 2009)

2 In Scotland much interesting work in TBA is being conducted by the Scottish Assessment

Is for Learning (AifL) group (Learning and Teaching Scotland, 2006) supported by the Ministry of Education in Scotland and involving many classrooms

(Butler, this issue; Curriculum Development Institute [Hong Kong],

2002; Ministry of Education [Singapore], 2008; Xu & Liu, this issue) as

well as in some developing countries, including South Africa, Ghana, and

Zambia (Pryor & Akwesi, 1998; Pryor & Lubisi, 2002) It is also actively

promoted in the United States (e.g., Popham, 2008a, 2008b; Stiggins,

2008; Stiggins, Arter, Chappuis, & Chappuis, 2007), although always

over-shadowed by national testing programs At the same time, English

lan-guage teachers are increasingly being called on to plan and implement

their own assessment instruments and procedures to monitor and

evalu-ate student progress in their classrooms, and new curriculum documents

and professional teaching standards increasingly demand English

language teachers be knowledgeable and skilled in TBA (see, e.g.,

TESOL, 2005)

However, despite this widespread embrace of various forms of TBA in

school and adult education, there has been comparatively little specifi c

research into the TBA of English as a second or additional language TBA

has been neglected by researchers partly because of the uncertain status

of TESOL as a discrete curriculum area in schools and tertiary

institu-tions, partly because of the traditional dominance of the fi eld by

large-scale English language tests such as the Test of English as a Foreign

Language (TOEFL) and the International English Language Testing

System (IELTS) and their research priorities and needs, and partly

because of the ongoing critique of notions of standard English and

mod-els of correctness as well as debates over native versus nonnative speaking

teachers and the implications for assessment

What TBA research that has been done in TESOL reveals much

vari-ability, a lack of systematic principles and procedures, and a dearth of

information as to the impact of TBA on learning and teaching In

Australia, several studies of the use of large scale criterion-referenced

English as second language (ESL) assessment frameworks in schools

(Breen et al., 1997; Davison & Williams, 2002) have revealed a great

diver-sity in teachers’ approaches to assessment, infl uenced by the teachers’

prior experiences and professional development, the assessment

frame-works and scales they used, and the reporting requirements placed on

them by schools and systems Concerns have also been raised about, on

the one hand, the ad-hoc or impressionistic nature of many teacher

judg-ments (Leung, 1999; Leung & Teasdale, 1997) and, on the other hand,

mechanistic criterion-based approaches to TBA, which are often

imple-mented in such a way that they undermine rather than support teachers’

classroom-embedded assessment processes (Arkoudis & O’Loughlin,

2004; Black & Wiliam 1998; Carless, 2005; Davison, 2004; Leung, 2004a,

2004b)

Research into TBA in TESOL is further complicated by the

consider-able uncertainty and disagreement around the concept of TBA itself and

by its intrinsically co-constructed and context-dependent nature (Black & Wiliam, 1998; Brookhart, 2003; McMillan, 2003; McNamara, 2001; Stiggins, 2001) When the principles and procedures underlying TBA are not clear, the basis for research and development is even muddier, hence the need for more public and mainstream discussion of the issues This review article aims, fi rst, to defi ne more clearly the concept of TBA in English language teaching and second to explore some of the key con-ceptual issues and challenges for the fi eld, as well as the implications for practice The article concludes with a summary of some of the areas in which more research into TBA is needed

DEFINING TBA

There is no widely accepted common defi nition of teacher-based

assess-ment in the English language teaching fi eld, with many terms used

interchangeably to refer to the same practices and procedures,

includ-ing terms such as alternative assessment , classroom and/or school-based

assessment , formative assessment , and more recently, assessment for learning

Such terms highlight different aspects of the assessment process, but all tend to be used to signify a more teacher-mediated, context-based, classroom-embedded assessment practice, explicitly or implicitly defi ned in opposition to traditional externally set and assessed large scale formal examinations used primarily for selection and/or account-ability purposes Thus, for the purposes of this article we take TBA to

mean much more than just who is doing the assessing; TBA also has implications for the what , where , how and most importantly, the why of

It allows for the collection of a number of samples of student work

centre or examination hall

It is conducted by the students’ own teacher, not a stranger

•

It involves students more actively in the assessment process, especially

and learning programme

It complements other forms of assessment, including external

Defi ned in this sense, TBA shares many of the characteristics of

assess-ment for learning (AfL), a concept fi rst used in the United Kingdom in

the late 1980s, and widely promoted through the work of the Assessment Reform Group (Assessment Reform Group, 1999, 2001; Black & Wiliam, 1998) The term was introduced to ensure “a clear distinction be made

between assessment of learning for the purposes of grading and ing, which has its own well-established procedures, and assessment for

report-learning , which calls for different priorities, new procedures and a new

commitment” (Assessment Reform Group, 1999, p 2) The Assessment Reform Group (1999, p 7) has described AfL’s defi ning characteristics

how to take them

underpinned by confi dence that every student can improve

more formal planned assessments at the end of a unit, term, or year which are used to evaluate student progress and/or grade students In an

assessment of learning culture, formative and summative assessment are

seen as distinctly different in both form and function, with teacher and

assessor roles clearly demarcated, but in an assessment for learning

cul-ture, it is argued that even summative assessments of the students’ guage skills can and should also be used formatively to give constructive student feedback and improve learning (see, e.g., Biggs, 1998; Carless, 2008; Davison, 2007, Davison & Hamp-Lyons, 2009; Hamp-Lyons, 2007; Harlen, 2005; Kennedy et al., 2006; Taras, 2005) These researchers argue that provided summative assessment is undertaken while students are still learning (and teachers are still teaching), such assessments can and

lan-should also be used for formative purposes, that is, to improve learning

and teaching, thus building a more coherent and stronger assessment for

learning culture Kennedy et al propose that in this more inclusive model

of assessment:

1 All assessment needs to be conceptualized as assessment for learning

2 Feedback needs to be seen as a key function for all forms of

assessment

3 Teachers need to be seen as playing an important role not only in

relation to formative assessment but in all forms of summative

assess-ment as well—both internal and external

4 Decisions about assessment need to be viewed in a social context

because in the end they need to be acceptable to the community

Kennedy concludes that “the continuing bifurcation between formative

and summative assessment is no longer useful, despite the fact that such

a distinction has resulted in some excellent research and development

work on formative assessment” (p 14) He joins Harlen (2005), Carless

(2008), and others in calling for more research to be conducted into

summative assessment, and as Carless puts it, tests as “productive learning

opportunities” (p 8) However, Kennedy challenges Roos and Hamilton’s

(2005) view that summative assessment as a procedure is too deeply

entrenched, in Roos and Hamilton’s words, to become “a valid activating

mechanism for goal-directed educational activities” (p 7) Biggs (1996,

1998) also argues that an exclusive focus on formative assessment may

leave many negative summative assessment practices uncontested He

points out that this is deeply problematic given summative assessment’s

signifi cant infl uence on student learning, often negative backwash

under-mining any of the positive impacts of formative assessment In fact, as has

been well-documented in systems such as Hong Kong and Singapore

(e.g., Cheah, 1998; Hamp-Lyons, 2007), it is extremely diffi cult to sustain

any signifi cant teacher-based formative assessment practices in most

tra-ditional examination-dominated cultures 3

The traditional concept of formative assessment also needs to be

prob-lematized In AfL, formative assessment is seen as having two key

func-tions: informing and forming That is, formative assessment not only

shapes the decisions about what to do next, by helping the teacher to

select what to teach in the next lesson, or even in the next moment in the

3 In Hong Kong studies of the implementation of teacher-based assessment innovations such

as the Target-Oriented Curriculum in primary schools (e.g Cheung & Ng 2000; Carless

2004; Adamson & Davison 2003) and the Teacher Assessment Scheme in senior secondary

science (Yung, 2006) found that any change in teacher assessment practice was diffi cult,

severely constrained by traditional school culture and by teacher, parent, and student

expectations

lesson; the student also has to understand what they have learned and what they need to learn next (Black, 2001; Black, Harrison, Lee, Marshall, & Wiliam, 2003a, 2003b; Black & Wiliam, 1998) The learner’s role is cru-cial because it is the learner who does the learning This point seems obvious, even trite, but it is central to the AfL philosophy and, if treated seriously, clearly highlights where formative assessment can go wrong As Torrance (1993) argued some years ago, many teachers are at risk of assuming formative assessment is at best “fairly mechanical and behav-iouristic in the graded test tradition”; at worst summative, “taking snapshots of where the children have ‘got to’, rather than where they might be going next” (p 340)

Teachers coming from more traditional assessment cultures make two common misinterpretations of formative assessment First, there is a widespread assumption that any continuous assessment is by defi nition formative, but this is not necessarily the case—a series of weekly tests are continuous, but they are not formative if they are not used by students to improve their learning:

The term ‘formative’ itself is open to a variety of interpretations and often means no more than that assessment is carried out frequently and

is planned at the same time as teaching Such assessment does not sarily have all the characteristics just identifi ed as helping learning It may

neces-be formative in helping the teacher to identify areas where more tion or practice is needed But for the pupils, the marks or remarks on their work may tell them about their success or failure but not about how

explana-to make progress explana-towards further learning (Assessment Reform Group,

1999, p 7)

Second, there is a common misconception that a so-called alternative form of the assessment automatically makes it formative; that is, assess-ments like portfolios and oral presentations are by defi nition formative However, such assessments can be and sometimes are components of large-scale externally set and assessed examinations, for example, the ubiquitous external oral examinations of many Asian educational systems which are not used at all for formative purposes

To summarize, then, in an AfL culture, TBA needs to be continuous and embedded naturally into every stage of the teaching–learning cycle, not just at the end (see Ministry of Education [Singapore], 2008, for an example of this in a K–12 curriculum) All assessments (even those for accountability purposes) need to be designed and implemented with the overriding aim of improving student learning, with AfL as the domi-nant educational ethos In such a classroom or institution, teachers would be continually engaged in various forms of formative assessment, even at the end of the course In this model of TBA, outlined in Table 1 ,

T

modeled on summative assessments but used for for

during the course of the year tailored to the needs of the individual students and class

to needs of students but also to the demands of exter

student progress and gap between what should be and is

may involve multiple and varied sources e.g., self, peers, teacher

evaluation and extensive teacher feedback

elicit/check understanding; oppor

assessment includes not only the formal planned moments when dents undertake an assessment task but also the far more informal, even spontaneous moments when teachers are monitoring student group work and notice one student speaking more confi dently or another fail-ing to take an offered turn Because the goal of TBA is to improve stu-dent learning, self and peer assessment are an integral component of all assessment activity Feedback is also a defi ning element, with opportuni-ties for constructive and specifi c feedback related to specifi c assessment criteria and curriculum goals and content regularly reviewed by students and teachers

Such an integrated approach to assessment underpinned the recent development of a school-based assessment (SBA) component in the Hong Kong Certifi cate of Education Examination (HKCEE) in English Language (Davison, 2007; Davison & Hamp-Lyons, 2009) The stated purpose of the SBA component was to provide a more comprehensive appraisal of Forms 4–5 (Grades 9–10) learners’ achievement by assess-ing learning objectives which could not be easily assessed in public examinations while at the same time enhancing teaching and learning The initiative marked a shift from traditional norm-referenced exter-nally set and assessed examinations toward a more student-centered TBA system that drew its philosophical basis from the assessment for learning movement discussed earlier Teachers are involved at all stages

of the assessment cycle, from planning the assessment programme to identifying and developing appropriate formative and summative assessment activities right through to making the fi nal judgments In-class formal and informal performance assessment of students’ authentic oral language skills using a range of tasks and guiding ques-tions and the use of teacher judgments of student performance using common assessment criteria are innovative aspects of the new SBA, as

is the insistence that students play an active role in the assessment cess and the vigorous promotion of self and/or peer assessment and feedback (for a fuller discussion, see Davison, 2007; Davison & Hamp-Lyons, 2009)

Such TBA is assumed to have a number of advantages over external examinations, especially in assessing language, because effective lan-guage development requires not just knowledge but skill and application

in a wide range of situations and modes of communication Hence, like other performance-based subjects such as music, art, drama, and various vocational subjects, it is often argued that languages are better assessed through more authentic-like, performance-based assessments Table 2 summarizes some of the common advantages attributed to TBA com-pared with external examinations

However, a number of these claims made for the effi cacy, or even riority, of TBA over traditional assessments, especially those relating to

TABLE 2 Advantages of TBA Compared With External Examinations for Oral Language Assessment

(Adapted From SBA Consultancy Team, 2005)

Characteristics of classroom-based TBA Characteristics of exams Scope Extends the range and

diversity of assessment collection opportunities, task types, and assessors

Much narrower range of assessment opportunities: less diverse assessment; one exam per year

Authenticity Assesses work being done within

the classroom; less possibility of cheating as teacher knows student capabilities; assessments more likely to be realistic

Removes assessment entirely from teaching and learning;

stressful conditions may lead to students not demonstrating real capacities

Validity Improves validity through

assessing factors that cannot be included in public exam settings

Limits validity by limiting scope

of assessment, e.g., diffi cult to assess interaction skills in exam environment

Reliability Improves reliability by having

more than one assessment

by a teacher who is familiar with the student; allows for multiple opportunities for assessor refl ection/

standardization

Even with double marking, examiners’ judgments can be affected by various factors (task diffi culty, topic, interest level, tiredness, etc.), but little opportunity for assessor refl ection/review Fairness Fairness is achieved by following

commonly agreed processes, outcomes and stan dards; teacher assumptions about students and their oral language levels are made explicit through collaborative sharing and discussion with other teachers

Fairness can only be achieved by treating everyone the same, i.e., setting the same task at the same time for all students

Feedback Students can receive

constructive feedback immediately after the assessment has fi nished, hence improving learning

The only feedback is usually a grade at the end of the exam;

generally no opportunities for interaction with assessor; no chance to ask how to improve Positive washback

in general

Examinations by their nature can only be purely summative, and do not serve any teaching- related purpose; effects on teaching and learning may even be negative; may encourage teaching to the test and a focus on exam tech- nique, rather than outcomes Teacher and

student

empowerment

Teachers and students become part of the assessment process;

collaboration and sharing

of expertise takes place within and across schools

Teachers play little to no role in assessment of their students and have no opportunity to share their expertise or knowledge of their students;

students treated as numbers

(Continued)

Characteristics of classroom-based TBA Characteristics of exams Professional

development

Builds teacher assessment skills, which can be transferred to other areas of the curriculum

Teachers have no opportunity to build their assessment skills; get little or no feedback on how to improve as teachers Practicality

and cost

Once teachers are trained, TBA

is much cheaper as integrated into normal curriculum;

undertaken by class teacher as part of everyday teaching;

avoids wasting valuable teaching time on practice tests

Language assessment as currently practiced is very expensive in terms of task development (especially as multiple stimulus material is often needed to avoid cheating), assessor training and moderation and teaching/learning time

TABLE 2 (Continued)

validity and reliability, need to be explored further because they raise important theoretical issues that go to the heart of TBA and when applied

to the English language teaching fi eld

CONCEPTUAL ISSUES AND CHALLENGES IN TBA IN ENGLISH LANGUAGE TEACHING

There are obviously many issues and challenges confronting English language education in its movement toward greater use of high-quality TBA, ranging from the very practical concerns associated with any sig-nifi cant change in classroom practice, including the need to develop confi dence to overcome the inevitable implementation dip through to the more technical problems associated with learning how to construct explicit assessment criteria and tasks appropriate for range of individ-ual student needs (e.g., Fox, 2008), through to the signifi cant chal-lenge of changing deeply entrenched sociocultural attitudes and expectations (for a fuller discussion, see Brindley, 1995; Davison, 2007, Rea-Dickins, 2008) However, in this article we are more concerned with conceptual issues, in which questions of validity and reliability are central

As others have pointed out (e.g., Rea-Dickins, 2007), in TBA there is much debate over evaluation criteria with researchers such as Leung (2004a, 2004b) and Teasdale and Leung (2000), on one hand, arguing that the evaluation criteria traditionally associated with psychometric testing such as reliability and validity need to be reinterpreted in TBA, particularly in relation to in-class contingent assessment in interaction, but testers such as Clapham (2000) insisting that traditional test criteria

do apply to alternative TBA: