EFL teachers’ views of english language assessment in higher education in the united arab emirates and kuwait

EFL Teachers’ Views of English Language Assessment in Higher Education in the United Arab Emirates and Kuwait SALAH TROUDI University of Exeter Exeter, England CHRISTINE COOMBE Dubai Men’s College Dubai, United Arab Emirates MASHAEL AL-HAMLIY University of Kuwait Kuwait City, Kuwait Ⅲ Issues of assessment design and implementation in Kuwait and the United Arab Emirates (UAE) have attracted some attention over recent years, but teachers’ philosophies about assessment remain underexplored This article reports the findings of a qualitative study into the assessment roles and philosophies of a group of teachers of English as a foreign language (EFL) in the UAE and Kuwait Based on an open-ended questionnaire, the study showed that teachers’ views on the nature of assessment were informed by their knowledge of the field of language learning and teaching and by the contextual milieu and sociopolitical factors that govern their employment conditions The study also showed that teachers did not play a major role in assessment because of top-down managerial approaches to education and a concern for validity and quality assurance in large programmes Assessment continues to play a major role in learning and teaching and is extensively and intensively addressed in research studies and theoretical articles both in mainstream education and TESOL/TEFL literature However, the majority of published work in testing is still informed by psychometric views of assessment The view of assessment espoused in this article does recognise elements of validity and reliability of the psychometric model, but it goes beyond them to view assessment as an act of social and cultural practice and the product of a complex interaction of political, educational, economic, and historical factors and agendas (Pennycook, 1994, 2001; Shohamy, 2001) We define assessment as “the process of collecting information about a student to aid in decision making about the progress and language development of the student” (Cheng, Rogers, & Hu, 2004, p 363).With this practical definition, 546 TESOL QUARTERLY however, we also believe that assessment is an exercise of power that is caught up in an array of issues about testers’ and test-takers’ voices, roles, and beliefs This critical view of assessment, with an agenda of understanding and social change, suggests that teachers, students, and other stakeholders “construct the assessment knowledge by trying to make sense of the knowledge in a dialogical and co-operative way” (Shohamy, p 136) Although little research is being conducted into teachers’ beliefs and attitudes toward assessment practices and how these are shaped by cultural and institutional contexts (Davison, 2004), some recent research studies have started exploring the role of the English as a foreign language and English as a second language (EFL/ESL) teacher and his/her views of what constitutes an enriching assessment practice Cheng et al (2004), for example, report multifaceted and complex roles played by university instructors in Canada, Hong Kong, and China This large-scale comparative study found that teachers’ practices varied in three major areas: assessment purposes, methods, and procedures These differences, within and across settings, were due to varying cultural, institutional, and contextual factors; the nature of the courses; teachers’ knowledge of assessment; teaching experience; students’ needs; and the role of “external testing on teaching and learning” (p 378) In another comparative study by Davison (2004), 24 secondary school teachers in Hong Kong and Australia were asked about their views and interpretations of the construct being assessed, with specific reference to written argument and whether they felt that their judgements were trusted and legitimated in their communities The qualitative data revealed that teachers’ practices and orientations can be classified along a cline “from assessor as technician, to interpreter of the law, to principled yet pragmatic professional, to arbiter of ‘community’ values, to assessor as God” (p 324) This variety in interpretation of roles was also linked to the effect of assessment approaches such as norm-, criterion-, or construct-referenced on teachers’ views and how they interact with teachers’ knowledge Thus far, several studies have looked into teachers’ assessment practices, with a few focusing on teachers’ beliefs and knowledge affecting their decision-making processes in classroom-based assessment (Chang, 2005; Davison, 2004) No research of this kind, however, has been conducted in the Gulf region So far, all studies have concentrated on either the psychometric and quantitative element of assessment with a focus on students’ performance (e.g., Addamigh, 2006; Gamaroff, 2006) or on a number of factors involved in the testing of the four language skills (e.g., Al-Busaidi, 2007; Al-Hamly & Coombe, 2005; Lanteigne, 2008) Farah’s (2007) study on the effects that a high-stakes international test has on students’ access to a field of study of their choice is the only research with a BRIEF REPORTS 547 critical agenda aimed at questioning certain assessment practices in the Gulf region Looking into issues of consequential and ethical validity, Farah uncovered some of the detrimental effects that some major tests have on students’ lives The study also found that teachers did not have a voice and had no choice but to use externally imposed criteria to assess their students Continuing with this particular interest in teachers’ voice, the current study aims to redress a major gap in the assessment literature in the Gulf Informed by a sociocultural perspective and located within a broad interpretive and participatory framework, this study also draws on work done in the area of critical language testing to raise such issues as voice and inclusion in assessment decision making about test format, content, and structure This framework allows us to investigate teachers’ roles and views of assessment in the UAE and Kuwait The unique situation of the expatriate English language teacher (ELT) in these two countries is likely to play a significant role in the assessment scene THE STUDY The study sought to address the following main questions: What are teachers’ roles in student assessment in the UAE and Kuwait, and what are teachers’ assessment philosophies? With an interpretive framework of research that seeks to “develop an in-depth and interpreted understanding of the social world” (Ritchie & Lewis, 2003, p 22) by focusing on the individual as the main source of this interpretation, we developed an open-ended questionnaire to assist us in collecting rich qualitative data (Holliday, 2001) For logistical reasons it was not possible to conduct interviews with participants, who worked in different parts of the UAE and Kuwait We therefore developed a set of open-ended questions which we sent via e-mail to a number of tertiary institutions where we thought teachers interested in the issue of assessment were likely to participate in the study This purposive sampling technique is appropriate for the aims of this exploratory qualitative study (Dörnyei, 2007) The questions developed for the open-ended qualitative questionnaire (see appendix) were contextual, explanatory, and generative in nature They were meant to help us identify what exists in terms of assessment practices in the social world of the participants, explain why phenomena occurred, examine reasons and forces behind their occurrence, and generate and develop new conceptions or understanding (Richards, 2003; Ritchie & Lewis, 2003) The nature of the questions allowed the participants to express their views, and in some cases they replied in considerable detail 548 TESOL QUARTERLY The Participants A total of 21 tertiary ELTs replied to our invitation to take part in this study We followed established ethical research procedures, and all participants gave us their consent to use the data for publication and oral dissemination purposes Pseudonyms were used to protect the identities of the participants, who represented a total of nine colleges and universities All of them are expatriate teachers from a wide range of countries such as Sudan, the United Kingdom, the United States, and Egypt, working for fixed and renewable contracts The majority hold master’s degrees in TEFL/TESL or applied linguistics, and three hold doctorates; their teaching experience ranges from to more than 20 years The participants work in a variety of English for academic purposes (EAP) programmes which prepare students for their transition to university departments where subjects are taught through the medium of English, a common educational policy in the Gulf (Troudi, 2007) Three participants work in English for specific purposes (ESP) programmes Data Collection and Analysis Given our personal views on assessment, our familiarity with many of the institutions where the participants worked, and our critical agenda, we sought to minimize the impact of our views on the analysis of the data We therefore attempted to use a “strategic and technical detachment” approach to both data collection and analysis (Holliday, 2001, p 178) To avoid imposing our views on the data, we analyzed it using exploratory content analysis The emerging themes were categorised and codified and then compared with the whole set of data using a constant comparison method that included reading and rereading within and across the responses of the participants (Lalik & Potts, 2001) The analysis revealed a number of recurrent themes, minor categories, and even some individual occurrences However, it is beyond the scope of this article to include all the findings, such as the variety of practices in the two countries FINDINGS AND DISCUSSION The participants invoked a number of varied personal views toward assessment, but there were also some points of similarity, which are discussed in the following sections BRIEF REPORTS 549 Teachers’ Roles: Exclusion Versus Inclusion One recurrent theme was that teachers were not involved in assessment decision making about matters of approach and design Some teachers did not participate in the process of designing assessment tools which dealt with areas of content, format, number of test items, and marking rubrics Some teachers also reported that when institutional changes in assessment occurred, teachers were often excluded and their views were not solicited A number of EAP programmes had a structured and hierarchical approach to assessment planning and implementation Testing committees or supervisors were in charge of developing policies and assessment tools because teachers were perceived not to have expertise in this area In fact, in some cases teachers were not informed about the content of midterm or final tests and had to wait until the testing day to discover this with their students This element of secrecy caused many teachers to feel distrusted and disrespected One of our teacher informants, Ahmed, considered that “teachers should be given more power in assessing their own students in a way that meets students’ needs and future/expected job demands,” and his opinion was echoed by others in different institutions Because these programmes have large numbers of EFL teachers, efficiency, practicality, and reliability require that assessment be centrally managed, which allows only a few teachers to be directly involved in decisions on assessment Another explanation is that teacher evaluation is linked, although not officially, to students’ performance, so there is a concern that teachers might be inclined to assist their own students to achieve high grades if they were involved in writing tests This distrust was mutual: Some of the excluded teachers felt that the problem was that, as one of the informant teachers put it, “there was very little to no expertise and/or training It is often done by people who think that they know what they are doing but in reality have no idea of basic testing theory.” Employment conditions and policies are at the root of such an atmosphere of distrust, for expatriate teachers are all contractually employed, and teacher evaluation and, in some cases, students’ grades may affect contract renewal Teachers’ Philosophies Alternative Assessment and Fairness Teachers were aware that one way of ensuring fairness for students was to use multiple methods in assessing their work Although they recognized the role of traditional or established measures like standardized tests, they believed that learners would benefit from a variety of 550 TESOL QUARTERLY instruments, as stated by Paul, who teaches in one of the institutions in the UAE: I believe that there should be a distinct difference between testing and assessment While I believe in various forms of traditional testing methods and the value of them in ESL/EFL, I also believe in the importance of alternative assessment techniques Students, in my opinion, should not always be merely tested on a regular basis or at the end of a particular unit of study Instead, they should be regularly assessed on their ability to use the language in real and meaningful situations Alternative assessments are ideal in these instances Students should also be given more autonomy in their particular learning situations and alternative assessments such as peer assessment, portfolios and rubrics/rating scales are excellent for accomplishing these goals Some participants invoked the issue of fairness and viewed authenticity and multiple-measures assessment as an approach to ensure that tests were fair and that students were not unfairly disadvantaged The students’ interests were at the centre of their concern In fact, one of the teachers noted that “every effort should be made within an assessment system to make the assessment of students fair.” In this context, performance-based assessment was suggested as a viable approach Mohamed, who teaches EAP to first-year students, explained that “teachers needed to be trained to implement performance-based assessment in a way that capitalises on its strength, i.e., validity, and minimises its weakness, i.e., reliability.” Initially, some views seemed to be presented in a culturally detached manner without references to the local and cultural contexts where the participants worked On further exploration, we could see that teachers’ assessment orientations were influenced by internal and external criteria as well as by their knowledge of their students and the educational and cultural contexts Elaborating on his own philosophy, Mohamed stated that “this personal philosophy is adopted as long as the context is teaching relatively smaller classes to meet specific job and/or community expectations.” We would add that teachers’ views are also influenced by their own educational principles and pedagogical experience This positioning confirms observations made by Davison (2004), who argues that within a student-based assessment approach, teachers consider “not only common assessment criteria and community constructs, but also the learner and the context” (p 326) No Assessment Philosophy One significant finding in this study is that some participants questioned the idea of a philosophy of assessment (or any abstract principles) BRIEF REPORTS 551 They were more concerned about their daily assessment practices and their impact on students In fact, Salwa in Kuwait explicitly admitted to not having a particular philosophy: “I don’t have a philosophy about assessment I mean what does that mean? Testing and assessment is a part of studying that I don’t question much—beyond issues of validity and reliability in individual tests I am involved with.” However, she raises a major concern about the nature of assessment and what it means to assess a student’s language ability In addition, although teachers recognised the intellectual debate about the authenticity and efficiency of assessment, they not see a viable alternative They focus therefore on how to develop assessment tools that best serve the interests of the students Assessment was seen as a learning opportunity and not only as a demonstration of proficiency These concerns are some of what Shohamy (2001) views as the main aims of the critical language-testing agenda Teachers did not explicitly invoke students’ rights and testers’ obligations, but they repeatedly stated their attention to and concern about the quality of their students’ educational experience Technical and Pedagogical Concerns The focus on the pedagogical and the practical side of assessment was echoed by the majority of teachers In the words of one participant, testing is a “complex matter [for] there is not a one-size-fits-all solution”; students may prefer different assessment methods While aware of the importance of reliability and content validity, some teachers stressed that “tests should be a motivating factor for students and, to achieve this, [a test] must be designed to be a learning as well as assessment experience” (see Leung & Mohan, 2004) It was also clear from a number of responses that teachers were speaking their minds regardless of institutional practices and mechanisms in place Their formal knowledge, experience, and educational principles informed their views and their interest in the educational experience of their students These views often clashed with some of the assessment practices put in place by testing committees and programme directors, as expressed by Pamela: Tests should be designed so that the format (question types, test structure, test timing, etc.) doesn’t get in the way of the students showing their “true” abilities/proficiency/achievement In light of that, students should be exposed, several times at least, to whatever format is chosen Moreover, the tests must reflect the test specifications which, in turn, reflect the curriculum designed in light of students’ needs In this study, contrary to Rea-Dickins’ (2004) observation that teachers often prioritize formal procedures and “underplay the observationdriven approaches to assessment which are strongly in evidence in this 552 TESOL QUARTERLY everyday classroom practice” (p 249), teachers were clearly aware of the limitations of the current formal procedures and were willing to challenge them CONCLUSION The analysis revealed a gap between teachers’ philosophies and their practices Overall, teachers were not involved in assessment-related decisionmaking processes They had little voice in this important element of the curriculum and were marginalised within a top-down managerial approach to assessment Most teachers wanted a more effective role and expressed frustration at being ignored This echoes findings in the literature that teachers find themselves at the confluence of different assessment cultures and faced with significant dilemmas in their assessment practices: sometimes torn between their role as facilitator and monitor of language development and that of assessor and judge of language performance as achievement (Rea-Dickins, 2004, p 253) With years of experience in the Gulf, many of these teachers have developed substantial knowledge of the culture of their students and their ways of learning It is this knowledge and experience that institutions need to capitalise on if the situation is to change to a more conducive environment for teaching and learning Local expertise needs to be recognised and continuously supported One way of doing so is by showing more professional respect to teachers’ knowledge and views and by providing them with professional development opportunities Despite being mostly voiceless in decisions on assessment matters, the participants in this study voiced major concerns about the nature of assessment While recognizing the importance of standardized and reliable measurement, they stressed the role of classroom-based teacher assessment that can be used for learning purposes and called for its incorporation into the curriculum THE AUTHORS Salah Troudi teaches applied linguistics and language education at the Graduate School of Education at the University of Exeter, England His research interests are in the areas of teacher education, critical applied linguistics, and language policies He coordinates the doctor of education in TESOL program in Dubai Christine Coombe has a doctorate in foreign/second language education from the Ohio State University in the United States She is currently a faculty member at Dubai Men’s College in the United Arab Emirates Her research interests are assessment, teacher evaluation, teacher effectiveness and leadership skills in English language teaching BRIEF REPORTS 553 Mashael Al-Hamly is an associate professor of applied linguistics at Kuwait University, Kuwait She has a doctorate in computer-assisted language learning (CALL) from the University of East Anglia in England She has published in regional as well international journals in the areas of CALL, testing, and translation studies REFERENCES Addamigh, K (2006) Construct validity of foreign language tests In C Coombe, P Davidson, & D Lloyd (Eds.), Proceedings of the 7th & 8th Current Trends in English Language Testing Conference (pp 55–72) Dubai: TESOL Arabia Publications Al-Busaidi, S (2007) Assessing the active and passive vocabulary knowledge of EFL students in the Sultanate of Oman In A Jendli, S Troudi, & C Coombe (Eds.), The power of language: Perspectives from Arabia (pp 114–127) Dubai: TESOL Arabia Publications Al-Hamly, M., & Coombe, C (2005) To change or not to change: Investigating the value of MCQ answer changing for Gulf Arab students Language Testing, 22, 509–531 Chang, C (2005) Oral language assessment: Teachers’ practices and beliefs in Taiwan collegiate EFL classrooms with special reference to Nightingale University Unpublished doctoral dissertation, University of Exeter, England Cheng, L., Rogers, T., & Hu, H (2004) ESL/EFL instructors’ classroom assessment practices: Purposes, methods, and procedures Language Testing, 21, 360–389 Davison, C (2004) The contradictory culture of teacher-based assessment: ESL teacher assessment practice in Australia and Hong Kong secondary schools Language Testing, 21, 305–334 Dörnyei, Z (2007) Research methods in applied linguistics Oxford: Oxford University Press Farah, B (2007) Students’ voices on their experiences with a high stakes language test In S Midraj, A Jendli, & A Sellami (Eds.), Research in ELT contexts (pp 252– 270) Dubai: TESOL Arabia Publications Gamaroff, R (2006) What test scores mean? In C Coombe, P Davidson, & D Lloyd (Eds.), Proceedings of the 7th & 8th Current Trends in English Language Testing Conference (pp 87–89) Dubai: TESOL Arabia Publications Holliday, A (2001) Doing and writing qualitative research London: Sage Lalik, R., & Potts, A (2001) Social reconstructivism as a framework for literacy teacher education In C M Roller (Ed.), Learning to teach reading: Setting the research agenda (pp 119–135) Newark, DE: International Reading Association Lanteigne, B (2008) Using advertisements in test item writing In A Jendli, C Coombe, & S Troudi (Eds.), Best practices in English language teaching (pp 345– 354) Dubai: TESOL Arabia Publications Leung, C., & Mohan, B (2004) Teacher formative assessment and talk in classroom contexts: Assessment as discourse and assessment of discourse Language Testing, 23, 335–359 Pennycook, A (1994) The cultural politics of English as an international language London: Longman Pennycook, A (2001) Critical applied linguistics: A critical introduction London: LEA Rea-Dickins, P (2004) Understanding teachers as agents of assessment Language Testing, 21, 249–258 Richards, K (2003) Qualitative inquiry in TESOL London: Palgrave Ritchie, J., & Lewis, J (2003) Qualitative research practice: A guide for social science students and researchers London: Sage 554 TESOL QUARTERLY Shohamy, E (2001) The power of tests: A critical perspective on the uses of language tests Harlow, England: Pearson Education Troudi, S (2007) The effects of English as a medium of instruction In A Jendli, S Troudi, & C Coombe (Eds.), The power of language: Perspectives from Arabia (pp 3–19) Dubai: TESOL Arabia Publications APPENDIX Open-Ended Questionnaire Please try to answer these questions in as much detail as you can Use as many additional pages as necessary How long have you been teaching English in the Gulf? What is your highest educational qualification? What is your personal philosophy about ESL/EFL assessment*? Please describe how students are actually assessed in your current professional context (please provide concrete examples) What grading system is in use in your institution (how is grade divided in terms of percentage? e.g midterm 30%, final 50%, projects 10%)? What you think of the current assessment practices in your institution? Do you have any suggestions about how students should be assessed? What are the teachers’ roles in assessment in your context? Ideally how you see your role in assessing students’ language? 10 Are there any assessment challenges that teachers face in your context? 11 What coping strategies you utilise when faced with some of the challenges you mentioned? * Assessment could include testing, measurement, and all other ways of evaluating students’ language BRIEF REPORTS 555 ... speaking their minds regardless of institutional practices and mechanisms in place Their formal knowledge, experience, and educational principles informed their views and their interest in the educational... to include all the findings, such as the variety of practices in the two countries FINDINGS AND DISCUSSION The participants invoked a number of varied personal views toward assessment, but there... that there should be a distinct difference between testing and assessment While I believe in various forms of traditional testing methods and the value of them in ESL /EFL, I also believe in the