Evaluation of washback effects of ESP end - term test for second-year students at UEB-VNUH on teachers at Faculty of English - ULIS - VNUH = Đánh giá tác động c

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES FACULTY OF POST-GRADUATE STUDIES PHẠM THU HÀ EVALUATION OF WASHBACK EFFECTS OF ESP END - TERM TEST

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES

FACULTY OF POST-GRADUATE STUDIES

PHẠM THU HÀ

EVALUATION OF WASHBACK EFFECTS OF ESP END - TERM TEST FOR SECOND-YEAR STUDENTS AT UEB - VNUH ON

TEACHERS

AT FACULTY OF ENGLISH –ULIS –VNUH

Đánh giá tác động của bài Kiểm tra Tiếng Anh Thương Mại cuối kỳ dành cho Sinh viên năm thứ 2 Trường Đại học Kinh Tế- ĐHQGHN đến việc giảng dạy của giáo viên Bộ môn Tiếng Anh Kinh Tế- Khoa Tiếng Anh

ĐH Ngoại Ngữ - ĐHQGHN

M.A MINOR PROGRAMME THESIS

Major: Methodology Major code: 60.14.10

Hanoi - 2010

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES

FACULTY OF POST-GRADUATE STUDIES

PHẠM THU HÀ

EVALUATION OF WASHBACK EFFECTS OF ESP END - TERM TEST FOR SECOND-YEAR STUDENTS AT UEB - VNUH ON

TEACHERS

AT FACULTY OF ENGLISH –ULIS –VNUH

Đánh giá tác động của bài Kiểm tra Tiếng Anh Thương Mại cuối kỳ dành cho Sinh viên năm thứ 2 Trường Đại học Kinh Tế- ĐHQGHN đến việc giảng dạy của giáo viên Bộ môn Tiếng Anh Kinh Tế- Khoa Tiếng Anh

ĐH Ngoại Ngữ - ĐHQGHN

M.A MINOR PROGRAMME THESIS

Major: Methodology Major code: 60.14.10 Supervior: Dr Dương Thị Nụ

Hanoi - 2010

TABLE OF CONTENTS

INTRODUCTION……… 01

1 Statement of the problem and rationale for the study……… 01

2 Aims of the study ……… 03

3 Scope of the study ……… 04

4 Research questions ……… 04

5 Research methodology ……… 04

6 Design of the study ……… 05

DEVELOPMENT ……… 06

CHAPTER I – LITERATURE REVIEW ……… 06

I.1 Language test and its purposes ……… 06

I.1.1 Definition of a language test ……… 06

I.1.2 Purposes of a language test ……… 06

I.2 Achievement Tests and English for Specific Purposes ……… 08

I.2.1 Definition and types of achievement tests ……… 08

I.2.2 English for Specific Purposes and English for General Business Purposes ……… 09

1.2.3 Characteristics of an ESP achievement test ……… 11

I.3 Washback effects of tests ……… 12

I.3.1 Definition of washback ……… 12

I.3.2 Types of washback ……… 12

I.3.3 Teachers and washback ……… 14

CHAPTER II - THE STUDY ……… 16

II.1 Background to the study: The context of English teaching, learning and testing of second year students at UEB – VNUH ……… 16

II.1.1 The context of English teaching and learning at UEB – VNUH ……… 16

II.1.2 Description of the language testing situation and the currently used ESP final achievement test at UEB – VNUH ……… 18

II.1.2.1 Description of the language testing situation at UEB- VNUH …… 18

II.1.2.2 The currently used ESP final achievement test at UEB- VNUH … 19

II.2 Methodology of the study ……… 21

II.2.1 Research approach ……… 21

II.2.2 Subjects ……… 22

II.2.3 Methods and procedures of data collection ……… 22

II.2.3.1 Methods of data collection ……… 22

II.2.3.2 Procedures of data collection ……… 23

II.2.4 Methods of data analysis ……… 24

CHAPTER III – FINDINGS, DICUSSION AND RECOMMENDATIONS 25 III.1 Findings from collected data ……… 25

III.1.1 Teachers’ responses to questionnaires and interviews ……… 25

III.1.2 Data analysis of classroom observation ……… 33

III.2 Discussion of findings: Washback effects of the ESP end-term test on teachers ……… 34

III.2.1 Teaching contents and teaching materials ……… 34

III.2.2 Teaching methodology ……… 35

III.2.3 Teachers’ attitudes and behaviors in class ……… 35

III.3 Recommendations to teachers to maximize the positive washback effects and minimize the negative washback effects of the ESP end-term test on their teaching ……… 36

CONCLUSION ……… 39

REFERENCES ……… 41 APPENDICES

ABBREVIATIONS

EFL: English Foreign Language

EGBP: English for General Business Purposes

ESP: English for Specific Purposes

ESBP: English for Specific Business Purposes

UEB: University of Economics and Business

ULIS: University of Language and International Studies

VNUH: Vietnam National University, Hanoi

LIST OF CHARTS

Chart 1: Teachers’ fulfillment of class teaching content to the syllabus ……… 28 Chart 2: Teachers’ following teaching process and conducting in-class tasks 29 Chart 3: Teachers’ teaching materials ……… 30 Chart 4: Teachers’ care for students’ past test scores ……… 32 Chart 5: Teachers’ anticipation of changes on their part when having detailed report of students’ past test scores ……… 33

LIST OF TABLES

Table 1: Curriculum frame for QHE 2008 students of UEB – VNUH ……… 17

Table 2: Assessment frame for QHE 2008 students of UEB-VNUH ………… 19

Table 3: Specification grid of ESP end-term test for second semester of

Table 4: Teachers’ belief on factors that influence teaching ……… 25 Table 5: Teachers’ knowledge of and attitudes towards the ESP final achievement test of the second semester ……… 26 Table 6: Teachers’ attitudes and behaviors in class ……… 31

INTRODUCTION

1 Statement of the problem and rationale for the study

From the early 1960s, English for Specific Purposes (ESP), which is defined by

Strevens (1988, p.1) as “a particular case of the general category of specific purpose

language teaching”, has grown to become one of the most prominent areas of English

Foreign Language Teaching Hutchinson and Waters (1987, p.19) see ESP as an approach rather than a product, by which they mean that “ESP is an approach to language teaching

in which all decisions as to content and method are based on the learner's reason for

learning" Also, learners’ needs, the typical feature of ESP, are highlighted by

Dudley-Evans, T., and St John, M J., 1998, p.123) with the belief that “ESP has as its basis in an

investigation of the purposes of the learner and the set of communicative needs arising from those needs” Since emphasis is laid on learners’ needs, in discussing ESP teaching, a

number of factors must be taken into consideration such as motivation, learning strategies, etc Also, due attention should be paid to issues closely related to any language courses such as syllabus design, materials development, methodology, assessment, etc From the perspective of Dudley-Evans and St John (1998) assessment does not stand alone, but occupies a prominent place in the ESP process, giving an ESP teacher a wealth of information on the effectiveness and quality of teaching and learning They also suggest that ESP testing yields an observed judgment of the effectiveness of teaching

It is generally recognized that in EFL teaching context in Vietnam, the teaching of Business English has received considerable attention in the last decade of the 20th century, especially in specific business schools or institutions Such implementation of Business English is visibly important as it can meet the demands of pre-experienced learners in terms of language skills related to business fields so that they can adjust themselves to the fast-changing business environment later on In the specific context of EFL teaching and

learning in VNUH, language needs of Business English of students at UEB-VNUH were

identified by the research carried out by the Division of English for Economics, Faculty of English, ULIS- VNUH in 2007 Findings from the research revealed that students expect to engage in the language with the specific aims of getting access to specialized Business English vocabulary, enlarging their knowledge of the subject matters related to commercial

English, becoming prepared for some common business situations (i.e going for an interview, conducting professional correspondence, taking the minutes, etc.) so that they can use the language in their prospective jobs

Regarding the features of such ESP teaching approach, and in order to meet the needs of second-year students at UEB- VNUH for their prospective jobs as businessmen or economists, for the past two academic school years, Division of English for Economics,

Faculty of English-ULIS-VNUH has moved from the self-complied Business English course book to a new set of Market Leader Intermediate (Cotton, Falvey, and Kent, 2005)

This content-based course book emphasizes the use of authentic materials and Business Communication skills development so as to encourage second-year students at UEB to take active roles in their class activities as well as improve and expand their four English language skills to a higher level Since putting in used, the course book has received great acceptance from both teachers and students for its easy-to-follow structure, stimulating contents and useful language specificity inputs

Such a change in teaching syllabus requires changes in assessment, since testing is closely related to what is taught and is subject to whatever changes taking place in course books Regarding this newly implemented English Business course book, the evaluation of students’ improvement is done through continuous assessment One component of this is the ESP end-term test, a kind of achievement test that second-year students are required to sit for at the end of the semester As the test accounts for 50% of the final score, it is supposed to affect the students in many aspects This achievement test is also supposed to

be meaningful since Hughes (2001, p.10) reasons that “achievement tests are directly

related to language courses, their purposes being to establish how successful individual students, groups of students, or the courses themselves have been in achieving objectives”

While such a test may have influences on students, it may at the same time affect the teachers in many ways such as teaching contents and materials, teaching methodology, attitudes and behaviors In their study in a Nepalese educational contexts, Herman and Golan (1993, cited in Chen, 2002, p.3) reported that over 50% of the teachers admitted that they would give substantial attention to mandated tests in their instructional planning and delivery In devising their syllabi for instruction, they would look at prior tests to assure that they covered the subject matter of the test or test objectives Shohamy et.al (1996), Cheng (1997) and Wantanabe (1996) also addressed teachers’ use of past test papers and

test-oriented textbooks in sessions near test time These researchers at the same time mentioned teachers’ feeling and attitudes towards testing process and test scores It is evident that there is a chance for a test to influence teachers, either positively or negatively However, expertise in assessment of Business English tests in EFL teaching context

in ULIS- VNUH is still a rarely researched area In fact, there has been little practical attempt to evaluate Business English tests For the past five years, teachers from English for Economics Division, Faculty of English, ULIS-VNUH, as test-makers have got opportunities to take part in conducting and delivering Business English end-term tests Although learners’ needs were identified, up to now there has been no concern paid to the

washback effect, the influence a test has on teaching and learning, in general and

washback effects of ESP end-term tests on teachers in particular, i.e how such tests influence teachers’ teaching, how positive washback can be maximized and negative washback can be minimized

Such gap has encouraged the researcher to choose “Evaluation of washback effects

of ESP end - term test for second-year students at UEB- VNUH on teachers at Faculty

of English-ULIS-VNUH” as the topic of her research It is hoped that this study would

make a modest contribution to better the teaching of Business English to second-year students at UEB-VNUH

2 Aims of the study

The study aims to primarily seek for evidences of washback effects of the ESP term test for second year students of UEB - VNUH on teachers at English for Economics Division, Faculty of English – ULIS -VNUH More specifically, it focuses on:

end-1) Investigating the washback effects that the ESP end-term test for second-year students has on teachers of English for Economics Division, Faculty of English - ULIS - VNUH;

2) Evaluating whether such evidences of washback (if any) are positive or negative to teachers;

3) Proposing ways to enhance the positive impacts and to minimize the negative effects of the test on teacher’s teaching of Business English

Practically, by looking for evidences of washback effects, the study aims at highlighting the close relationship between teaching and testing, and therefore, may be a source of reference in the attempt to better teaching and improve testing Business English

for second-year students of UEB-VNUH The study is also expected to be useful for the researcher’s colleagues and anyone who is concerned about the matter of testing in general and the interrelation of Business English teaching and testing in particular

3 Scope of the study

It would be too ambitious for this small-scaled study to cover all aspects of testing,

a broad field of language teaching methodology, within a short time and with limit reference materials Therefore, the study is limited to the washback effects of the ESP end-term test for QHE 2008 students at UEB -VNUH on teachers at English for Economics Division, Faculty of English, ULIS -VNUH

Also in this study, the test washback effects will be interpreted at a micro level within the classroom setting rather than being discussed at a macro level More specifically, it just focuses on the wash back effects of the ESP end-term test on teaching

Besides, the term “ESP” in this study is generally used as a conception of English for

General Business Purposes, or shortly Business English, whose target learners are experienced learners preparing to work in an occupational business context

5 Research methodology

The research methodology applied in this study is Survey Research This study was approached as a combination of both qualitative and quantitative methodologies, that is, to take advantages of survey questionnaires, observation and semi-structured interviews as the data collection tools The collected data are then analyzed quantitatively and qualitatively In addition, information from other sources such as Internet, journals, books

was collected and synthesized

Participants for the study are 12 teachers coming from English for Economics Division, Faculty of English – ULIS – VNUH as respondents to the survey questionnaire

Two teachers are randomly selected for class observation and subsequent semi-structured interviews

6 Design of the study

The study is divided into three main parts: Part I is the Introduction to the study Part II is the Development and Part III is the conclusion

In the Development, Chapter I reviews the literature on language testing,

achievement language tests and English for Specific Purposes achievement tests and

washback effects Chapter II addresses the testing context of Business English end-term

test for second-year students at University of Economics and Business, Vietnam National

University, Hanoi and describes the methodologies of the study Chapter III presents,

analyzes and synthesizes data collected from questionnaire survey, classroom observation and interviews and makes several suggestions to maximize the positive washback effects and minimize the negatives ones on the teaching of Business English to second-year

students Finally, the Conclusion part presents conclusions about the washback effects of

the test on teachers

References materials are listed along with appendices including the questionnaire for teachers, observation checklist, interview questions and the Business English end-term test used in the second semester of academic school year 2009-2010

DEVELOPMENT CHAPTER I – LITERATURE REVIEW

I.1 Language test and its purposes

I.1.1 Definition of a language test

Caroll (1968, p.46, cited in Bachman, 1995, p.20) considered a test is “a procedure

designed to elicit certain behavior from which one can make inferences about certain characteristics of an individual” According to Alderson, Clapham and Wall (1995, p.41),

a language test is a set of test items, each test item “consists of a method of eliciting

behavior or language, together with a system whereby that behavior or language can be judged” Along this line, Heaton (1990) holds that tests should be considered first as means

of assessing the students’ performance and then as devices to motivate students With regards to the belief that testing should serve the needs of teaching, Davies (1990, p.1)

considers that testing “provides goals for language teaching, and it monitors, for both

teachers and learners, success in reaching those goals”

In short, a language test is an instrument for assessing test-takers’ uses of language knowledge and skills It also plays the role of motivating device for students in their learning process and for teachers to adjust their teaching accordingly

I.1.2 Purposes of a language test

Testing contributes an important part in teaching and learning process It reflects teaching process and overall training objectives It also evaluates learners’ ability, suitability of teaching methods, teaching/ learning materials, teaching/ learning conditions

As testing, teaching and learning cannot be separated from each other, a language test must

be able to serve different purposes in teachers’ teaching and students’ learning

In a teaching process, the first and foremost reason for testing is to evaluate students’ knowledge and skills to use the target language Bachman (1995, p.55) points out

that “the fundamental use of testing in an educational program is to provide information

for making decisions, that is, to evaluate.” Tests are intended to discriminate between

those who have the ability and those who do not; and the results of tests provide teachers with necessary information to classify their students into different levels

The second purpose of testing is “to provide the teacher with information on how

effective his teaching has been” (Read, 1983, p.3) By administering a test to students, a

teacher can discover what her students have known and what they have not, measure their progress, and then decide what to teach next as well as which method should be applied to specific teaching situations Basing on students’ performance in those tests, teachers can also self-evaluate the effectiveness of their teaching methods, the appropriateness of the course objectives, the usefulness of the course book and the suitability of test items Such constructive feedbacks, in turn, urge teachers to adjust their future teaching activities

In addition, teachers write tests in order to provide additional materials for students

to practice further and check their knowledge and the ability of utilizing what they have learnt into different situations (Ur, 1996) As test tasks should be in a familiar form and able to cover the most important units of knowledge in the lessons, it can be considered as

a review of the periods leading up to the test “Testing should be firmly rooted in previous

classroom experiences in terms of activities practiced, language used, and criteria of assessment employed” (Weir, 1993, p.5) Consequently, by administering such tests,

teachers facilitate students’ awareness of the main points in their course and offer them one more chance to revise such points

In a learning process, tests are aimed at serving students in many ways Through their test performance and test results, students can check what they have been learning

and what they need to learn more A test, according to Read (1983, p.3) “can help both

teachers and learners to clarify what the learners really need to know.” In other words,

information from tests help students themselves reveal their strengths and weaknesses, so that they can determine appropriate types and levels of their future learning activities Accordingly, students may develop alternative learning strategies and/ or ask for help and support from teachers and peers in order to enhance what has already been good and improve what remains weak

Another purpose of testing is to motivate learners Tests encourage students to review specific materials (Ur, 1996) When the test is announced, students will have to pay more attention to what they have been taught as well as specific areas of knowledge related

to their specific field, i.e English for International Economics, English for Finance and Banking If they want to get good marks, they have to revise lessons and study harder Preparation for a test is, then, a process in which students spend time drilling materials in order to get good result, and thus achieve intended teaching/ learning objectives

In short, testing can be used to achieve many different aims in teaching and learning A test may be primarily designed for evaluating purposes but can latter be a useful means for promoting the effectiveness of the teaching-learning process

I.2 Achievement Tests and English for Specific Purposes

Language tests are classified in a number of ways In terms of testing method, language tests consist of oral and written ones Based on the methods of scoring and marking, they are divided into subjective tests and objective tests With regards to purposes, they can be categorized into proficiency, achievement, diagnostic and placement tests (Hughes, 2001, pp.7-19; Bachman, 1995, pp 71-77) The primary differences among these kinds of tests are in the purposes they serve and the manner in which their content is chosen As the scope of this study is washback effects of achievement tests, this part is focused on achievement tests

I.2.1 Definition and types of achievement tests

Different researchers have proposed different definitions of achievement tests but they all agree that this kind of test is used to evaluate how much students have achieve in their process of learning Brown (1994, p.259) defines an achievement test as the one that

“is related directly to the classroom lesson, units or even a total curriculum They are limited to particular materials covered in a curriculum within a particular time frame.”

This concept matches with Heaton’s (1990, p.4) one as he considers an achievement test

“measure a student’s mastery of what should have been taught (but not necessarily what has actually been taught) It is thus concerned with covering a sample (or selection) which accurately represents the contents of a syllabus or a course book.”

With reference to types, achievement tests are subdivided into progress achievement tests and final achievement tests

Progress achievement tests are conducted during the course to measure the progress that students are making They are conducted in some specific period during the course, depending on the syllabus, course objectives and testers’ decisions on when should measure the progress students are making In other words, progress achievement tests are supposed to help the teacher judge the degree of success of his/ her teaching as well as find out how much students have gained from what has been taught Accordingly the teacher can identify the weakness of learners or diagnose the areas not properly achieved so that he/ she can adapt other teaching approaches to attain better teaching and learning results

Final achievement tests, as their name suggests, are usually a formal examination, given at the end of the school year or at the end of the course to measure how far students have achieved the course objectives It may be written and administered by the Ministry of Education, official examining boards, or by members of certain teaching institutions Clearly the contents of this kind of test must be related to the teaching content and objectives concerned Correspondingly, Hughes (1990, p.11) suggests two approaches towards achievement test designing: syllabus-content approach and objective-content approach While the former can be considered a fair test as it is designed directly basing on what the students have learnt in the course book, the latter probably provides more accurate information about individual achievement and group achievement Both of the approaches are likely to promote backwash effect on teaching

I.2.2 English for Specific Purposes and English for General Business Purposes

Regarding the definition of ESP, Dudley-Evans and St John (1998) makes a distinction between three absolute characteristics and four variable characteristics of ESP The three absolute characteristics are that:

 ESP is designed to meet specific needs of the learners;

 ESP makes use of the underlying methodology and activities of the disciplines it serves;

 ESP is centered on the language (grammar, lexis, register), skills, discourse and genres appropriate to those activities;

The variable characteristics are as follows:

 ESP may be restricted to or designed for specific disciplines;

 ESP may use, in specific teaching situations, a different methodology from that of General English;

 ESP is likely to be designed for adult learners, either at tertiary level institution or

in a professional work situation It could, however, be used for learners at secondary school level;

 ESP is generally designed for intermediate or advanced students Most ESP courses assume basic knowledge of the language system, but it can be used with beginners From the above definition of ESP, it can be seen that contrary to the belief that ESP

is just the drilling of technical terms and grammatical structures, ESP is actually the viable

approach for enabling tertiary-level or adult language learners to efficiently acquire a sufficient level of mastery in the communicative forms of language required for their professional or occupational needs

ESP has traditionally been divided into two main areas: English for Academic Purposes (EAP) and English for Occupational Purposes (EOP) In terms of EOP, there are many subcategories and English for Business Purposes (or Business English) is one sub-

category Dudley-Evans and St John (1998, p.53) see Business English as “an umbrella

term used similarly to the term English for Specific Purposes to embrace both general courses in the appropriate lexis and grammar for business communication” As for their

classification, it includes English for General Business Purposes (EGBP) and English for

Specific Business Purposes (ESBP) The discussion in this study is concerned mainly with English for Business Purposes (or Business English) as EGBP whose target learners are adult learners preparing to work in an occupational business context, not an academic one

Unlike ESBP courses which are run for job-experienced learners who bring their own business context knowledge and skills to the language-learning situations, EGBP

courses are usually for pre-experienced learners or those at the very early stages of their

career They are similar to general EFL courses with the materials set in business contexts Many learners attend these courses at language schools and groups will usually be formed

on the basis of language level rather than job There is a range of good published materials for students and teachers to choose from, with input in text, audio and video formats The published course books are mainly designed for use on the extensive courses that have one

or two sessions a week, over several months or a year Most units contain work on the traditional four skills plus specific grammar and vocabulary development

The underlying construct of the course is often grammatical, as in the Macmillan

Business English Programme (Badger and Menzies, 1993, cited in Dudley-Evans, T., and

St John, M J., 1998, p.56) or in the light of Communicative Language Teaching as in

Market Leader Intermediate (Cotton, Falvey, and Kent, 2005) The focus is presentation,

through listening and reading, and grammar and vocabulary practice through exercises, which focus on accuracy There are also activities, tasks which are more open-ended and

develop fluency in all four skills The setting include “meeting people”, “making

arrangement”, “talking about company”, “traveling”, “doing business across cultures”,

etc Typical business-career content topics include marketing, branding, advertisement, product quality, product development, leadership and management styles, etc

Such courses teach a broad range of English through business settings rather than ESBP The vocabulary range of EGBP books clearly differs substantially from that of English for General Purposes but the language activities are core EFL ones and the answers are often predictable with more closed, right/ wrong responses than unpredictable, open responses

1.2.3 Characteristics of an ESP achievement test

Basically, ESP tests are no different in terms of qualities from other types of language tests In principle, any ESP test in general can be classified as a performance test assessing the skills needed to “perform” in the language successfully Test-takers’

performance on an ESP achievement test, therefore, depend largely on the interaction

between language knowledge and specific purpose content knowledge Douglas (2000,

p.10) cites that ESP tests are “contrived language use events” in which, ideally, the

test-taker’s language ability and knowledge of the specialized field are measured

ESP achievement tests are related in content, themes and topics to particular

disciplines, and involve a higher degree of language specificity More specifically, special

lexical, semantic and syntactic characteristics of technical language, in addition to its communicative functions must be exposed through the test In other words, an ESP achievement test is designed upon the demands of linguistic characteristics of the specialized area of work or study

Furthermore, one of the prevailing characteristics of any ESP achievement test is that it should contain tasks that mirror faithfully candidates' target language use situation Accordingly, ESP achievement tests are concerned to present learners with tasks that involve them in reading, listening to, speaking or writing the target language, and evaluating how well they can do this That is to say, the key to this assessment is to present learners with tasks that resemble in some way the sort of things they may have to do with

the language in real life In other words, the tasks must be authentic

Therefore, it is visible that the ESP achievement test, like the ESP test in general, is based on the analysis of learners’ target language use situations and specialized knowledge

of using English for real communication

I.3 Washback effects of tests

I.3.1 Definition of washback

The notion of “washback” is prevalent in language teaching and testing literature

Some writers like Hughes (2001, p.1) used the term backwash while washback was

preferred by Buck (1988, p.17, cited in Pan, 2009, p.258) and Bachman and Palmer (1996, pp.30-35) to describe the effects of testing on teaching and learning

Definitions of washback range from simple to complex ones While some

researchers see it as simple as “effects of the test on the classroom practice” (Berry, 1990, p.31, cited in Pan, 2009, p.259), or consider washback “refers to the fact that testing

controls not only the curriculum but also teaching methods and students’ learning strategies” (Biggs, 1995, p 3 cited in Pan, 2009, p.257), many others hold broader views

Pierce (1992, p.687, cited in Bailey, 1999, p.4) states that “washback effect refers to the

impact of a test on classroom pedagogy, curriculum development, and educational policy,” Bachman and Palmer (1996, pp.29-35) take a further step by stressing washback

as a subset of test impacts on society, educational systems, and individuals They further divided the test impact into two levels: the micro level (i.e., the effect of the test on individual students and teachers) and the macro level (the effects of the test on society and its educational systems)

In this study, a narrower interpretation of wash back will be adopted: wash back at

a micro level within the classroom; that is the effects a test has on teaching and learning More specifically, it focuses on wash back effects of an ESP test on teaching

Referring to washback effect of an ESP test, Hughes (2001) stated that “it seeks to

investigate the relationship between test use and the ESP situation in which it is used” Put

simply, it is the effect of ESP test on classroom instruction, on “what is taught and how it

is taught” (Dudley-Evans and St John, 1998, p.214) As for them, an ESP test, which

would be based directly on an analysis of English language needs of a specific group of learners as similar as possible to those which they would have to perform in real life, would probably be a more plausible case of beneficial backwash than a case in which no such analysis would be carried out

I.3.2 Types of washback

Generally, washback can be analyzed according to two major types: positive and negative, depending on whether it has a beneficial or harmful impact on educational

practices (Hughes, 2001) This section explores positive and negative wash back in terms

of the micro level – the wash back of tests on teaching and learning

In terms of positive wash back a test brings, it is obvious that teachers and learners will be motivated to fulfill their teaching and learning goals Tests induce teachers to cover their subjects more thoroughly, making them complete their syllabi within the prescribed time limits as well as motivate students to work harder to have a sense of accomplishment and thus enhance their learning Pearson (1988, p.107, cited in Pan, 2009, p.259) indicated

that “good tests can be utilized and designed as beneficial teaching-learning activities” so

as to encourage a positive teaching-learning process Furthermore, syllabus alteration or a new syllabus can be resulted from a creative and innovative test (Davis, 1985, p.18, cited

in Pan, 2009, p.259)

However, there exist a number of negative wash backs of a test First of all, the test will lead to the narrowing of content in the curriculum, focus of attention on those skills that are most relevant to testing Teachers tend to ignore subjects and activities that are not directly related to passing the exam, and tests accordingly alter the curriculum in a negative way Smith (1991, p.20, cited in Pan, 2009, p.260) points out that testing programs

“substantially reduce the time available for instruction, narrow curricular offerings and modes of instruction, and potentially reduce the capacities of teachers to teach content and to use methods and materials that are compatible with standardized testing formats”

Besides, measurement-driven instruction will definitely result in cramming, as Shohamy

(1992, p.15) indicated that “what students have learned is test language, instead of total

phases of understanding” Another negative washback is the placement of constraints on

teachers’ and students’ creativity and spontaneity, since many teachers detailed high

anxiety, fear and pressure to cover the materials, “as they felt that their job performance

was assessed by students’ test scores” (Shohamy et al., 1996, p.6) This will result in

disparage the professional judgment of educators as the tests may fail to create a correspondence between the learning principles and/or the course objectives to which they should be related

To summarize, in terms of the classroom setting, the positive washback integrates meaningful and innovative learning activities in teachers’ educational methodologies, and thus educators will devote more attention to students’ intentions, interests, and choices Students at the same time will be encouraged and motivated to work harder On the other

hand, the negative washback is that teachers will usually teach-to-the-test, narrow the curriculum and only focus on what will be tested Besides, cramming will be the negative

washback brought by measurement-driven tests

I.3.3 Teachers and washback

A glance at studies to date on washback effect of language tests shows that the most closely related area is test impacts on participants, namely test- takers, teachers, and

other “personnel involved in language teaching such as administrators, course designers,

materials developers” (Bailey, 1999, p.12) Each participant is influenced in a different

way, but due to the scope of this study, the next section only discusses how teachers can be influenced by washback

Bailey has conducted “it is safe to say that teachers are the most frequently studied

of all the participants in the washback process” (1999, p.18) Bachman and Palmer (1996)

persuasively argue that testing always has influence on teachers’ instruction; that is, if it appears obvious to teachers that they have to use a certain test, they may find it hard to

avoid “teaching to the test” (p.33) Meanwhile, there are cases when teachers themselves

feel unhappy with the results the test may produce, which reflects the unsatisfying quality

of the course Therefore, they require a change in testing procedure that can promote classroom’s instruction in order to enhance effective learning They then come to a conclusion that testing can influence teaching in a wide range, from almost nothing to quite

a lot, and can be both positive and negative

Likewise, six out of fifteen Alderson and Wall’s restatements of washback hypothesis are to emphasize the importance of teachers:

1 A test will influence teaching;

2 A test will influence what teachers teach; and

4 A test will influence how teachers teach;

7 A test will influence the rate and sequence of teaching; and

9 A test will influence the degree and depth of teaching; and

11 A test will influence attitudes to the content, method, etc of teaching and learning

(Alderson and Wall, 1993, pp 120-121) Researchers have observed classroom practices in various language learning and testing contexts and come to a variety of conclusions Among them are: teachers’ classroom behavior can either support or discourage the intended positive washback effect

of new or revised tests (Wall and Alderson’s case study in Sri Lanka, 1993); and test influence on experienced teachers may be different from that on novice ones, i.e new teachers’ lessons tend to have many additional activities while the experienced seem to fill their classes with more materials from the tests (Shohamy et al.’s observation in Israel, 1996) In many observed contexts, teachers change their teaching contents to adapt to a shift in the test rather than changing their methodology (Lam’s research in the Revised use

of English Test in Hong Kong, 1993; Cheng’s report on HKCCE, 1997; Wanatabe’s study

in Japanese context, all cited in Bailey, 1999) Similarly, Wanatabe (1996, p.31) strongly remarks three possible factors that might encourage or limit the washback on teachers:

1 the teachers’ educational background and/ or experience;

2 the differences in teachers’ beliefs about effective teaching methods;

3 the timing of the researchers’ observations

(Wanatabe, 1996, p.31, cited in Bailey, 1999, p.23) This comment shares the same viewpoints with which Lam (1993, cited in Bailey, 1999) holds He emphasizes that changing the test is not sufficient to draw out a change in

teachers’ methodology and learning outcomes, yet, “the challenge is to change the

teaching culture, to open teachers’ eyes to the possibilities of exploiting the test to achieve positive and worthwhile educational goals” (ibid., p.96, cited in Bailey, 1999, p.23)

In short, washback- one of the five criteria of a good test (namely validity, reliability, discrimination, practicality) has been thoroughly discussed by many researchers Their studies also provide general as well as specific guidelines on how to achieve these crucial decisive factors to the best It is the duty of testers to integrate them flexibly into real contexts so as to accomplish success in designing tests, which will result

in later successful teaching and learning

CHAPTER II - THE STUDY

II.1 Background to the study: The context of English teaching, learning and testing

of second year students at UEB - VNUH

II.1.1 The context of English teaching and learning at UEB - VNUH

 Students and their backgrounds

First and foremost, it is necessary to evoke the EFL learning context of students at UEB- VNUH by highlighting its main characteristics

University of Economics and Business is a young university (officially established

in 2007) with five training majors: Business Administration, Finance and Banking, International Economics, Political Economics, and Development Economics Students are trained to be business people or economists in the future

Students at UEB- VNUH, aged from 18 to 22, come from both rural and urban areas within the country and most of them had chances to get access to English, some had learned other languages such as Russian or French when they were at high school When starting classes with first year students, ESP teachers administer placement tests to diagnose students' abilities and distribute them among different language groups which differ in size from 20 to 25 students Particularly, the QHE 2008 second-year students were

already equipped with the background knowledge of General English with New Headway

Elementary, New Headway Pre-intermediate and New Headway Intermediate in the first

three semesters and in the 4th semester, they finished with the ESP course book Market

Leader Intermediate- New Edition, which is compulsory for all training majors

 The English teaching staff

The English for Economics Division belonging to the Faculty of English is a small division with only 13 teachers They take over teaching both General English and English for Specific Purposes majoring in Business English All the teachers have been well-trained in Vietnam and four of them obtained Master Degree in TESOL, the rest are doing their MA course in TESOL in ULIS -VNUH Although none of them had studied abroad, they continuously acquire for themselves a great deal of knowledge of General English and specialized subjects of Business English through their self-study and in-country training programs Most of them have a language teaching background and do not have first-hand

experience of the content and context of other disciplines or business nor have worked in business settings However, they are always fully aware of adapting suitable methods of teaching homogenous classes and applying technology in their teaching Business English

 The objectives of the English courses for non- English majors at UEB -VNUH

According to the program of the first stage education at the non-specialized language universities, and the decision No 477/TTG issued on September 5th, 1994 by the Prime Minister, non-English majors are to master basic knowledge of grammar, vocabulary, phonetics and some speech discourse to accomplish their competence in the target language However, under the decision No 2098/ DT made by VNUH and UEB - VNUH basing on the education purposes in training students to become businessmen or economists, the whole curriculum for QHE 2008 was framed as follows:

Credit

Semester

Number of hours of credit Course book

I 4 4 x 15 weeks = 60 crd hrs New Headway Elementary

II 4 4 x 15 weeks = 60 crd hrs New Headway Pre-intermediate

III 3 3 x 15 weeks = 45 crd hrs New Headway Intermediate

IV 3 3 x 15 weeks = 45 crd hrs Market Leader Intermediate

Table 1: Curriculum frame for QHE 2008 students of UEB - VNUH

In the first three semesters of General English, students work with New Headway

Elementary, New Headway Pre-intermediate, and New Headway Intermediate These

courses, which are designed for every student in non-specialized language universities in VNU, provide students with basic knowledge of grammar, pronunciation as well as develop and improve four kills of listening, speaking, writing and reading so that they can communicate general English in everyday life However, in the third semester, the teachers

at English for Economics Division, Faculty of English – ULIS- VNUH planted the seed for the next semester by preparing students with English for Specific Purposes through supplementary materials with vocabulary, reading passages related to Business English

In semester IV, the second year students of the academic school year 2009-2010

moved to Market Leader Intermediate (Cotton, Falvey, and Kent, published in 2005) to

master English for Business As clarified in the course syllabus, there are three main objectives to achieve after the course:

 Enriching learners’ language knowledge, i.e vocabulary and grammar related to business and economics through specialized themes such as career paths, marketing, e-commerce, customer services, management, ethics, etc

 Consolidating and improving learners’ language skills: communication and presentation skills, writing, reading and listening comprehension Specifically, communication skills refer to ability to communicate effectively in common business situations like negotiating, dealing with complaints/ conflicts, etc As for presentation skills, learners are expected to know how to make a presentation from preparing, delivering the presentation to answering questions Whereas writing focuses on forms of commercial correspondence such as letters, emails, minutes, memo, etc Besides, learners are provided with standard inputs, i.e articles, interviews, etc to improve receptive skills - reading and listening comprehension

 Developing other learning skills including team-work skills, discussion skills, problem solving skills through activities like Case study, role playing, etc so that students can practice and improve their English for Business and related fields Not less importantly, the course is also aimed at guiding them how to construct their

self-study skills so that they can apply such knowledge and skills in real life tasks

The course objectives described above were based on the analysis of students’ needs and expectations carried out by teachers before the implementing of teaching

Business English with Market Leader Intermediate Course book to students of QHE 2007

in the previous academic school year

II.1.2 Description of the language testing situation and the currently used ESP final achievement test at UEB – VNUH

II.1.2.1 Description of the language testing situation at UEB- VNUH

For QHE 2008 students, from semester I to semester IV, they had to undertake two kinds of tests: Progress Achievement Tests and Final Achievement Tests In order to obtain learners’ reliable progress and achievement, certain means of evaluation have been applied The following table will illustrate the assessment process more clearly:

Every week

Listening tests By teacher’s

decision Oral test/Presentation By teacher’s

decision Final

Table 2: Assessment frame for QHE 2008 students of UEB-VNUH

Most of the final achievements tests are constructed with the same frame – the core type of tasks that students have to complete the following parts:

1 Multiple choice in vocabulary and grammar

2 Reading comprehension

3 Sentences arrangement

4 Sentence transformation

5 Word formation

6 Mistakes identification and correction

7 Paragraph writing (in form of email letter or paragraph expressing viewpoints) However, within the scope of this study, the ESP final achievement test in 4thsemester will be described in more detail in the following section

II.1.2.2 The currently used ESP final achievement test at UEB- VNUH

The ESP final achievement test, in this case, the English for General Business Purposes test, is a syllabus-based achievement test whose content was taken from teaching

points in Market Leader Intermediate- New Edition course book delivered in second semester of academic school year 2009 -2010 The key features of the course book Market

Leader Intermediate (Cotton, Falvey, and Kent, 2005) should not be overlooked Some

main characteristics of this course book are described as follows:

 A theme-based course book that covers key themes in Business English such as products, e-commerce, business administration, commercial correspondence, etc

 Specific contents combined with grammar, functional language, subject- specific lexis of Business English are divided into six parts: Discussion, Vocabulary, Language Review, Reading, Skills and Case Study

 Eye-catching layout, lively illustrations, logical content arrangement with glossary and writing file are one of key factors that engage students in their learning

 Thoroughly updated materials from authentic sources such as the Financial Times help to develop students’ four skills as well as vocabulary related to the fast- changing business world

 Teachers’ Book and Practice File with reliable teaching guidelines and authentic tasks provide full support for the teaching and students’ self-study

The ESP final achievement test in semester IV for second-year students is based on the content of the course book It covers a wide range of knowledge in vocabulary, grammar, reading and writing skills related to Business English The table below describes this ESP end-term test with six parts and marking scale:

10 points

Mistakes correction

Information gap, Matching and Multiple choice

Requirements to complete the writing task

Paragraph writing (in form of email

to place an order)

20 points

Table 3: Specification grid of ESP end-term test for second semester of

academic school year 2009-2010 ( For specific test, see Appendix 4)

As mentioned above, this ESP final achievement test is aimed at assessing students’ performance in both language skills and content knowledge related to Business English More specifically, through inputs in forms of phrases, sentences mirrored in particular real-life-like tasks, the test could serve as a clue based on which the assessment of students’ language skills and content knowledge of Business English for the whole semester is targeted, ranging from reading skills, vocabulary, grammar, writing skills to Business communication skills Particularly speaking, it could, to some extent, meet the students’ needs in terms of getting involved in specialized Business English vocabulary, enlarging their knowledge of the subject matters related to commercial English

Besides, as the test was based on the content of the course book, it reflected the principle of teacher’s constructing a test proposed by Hutchinson and Waters (1987) as

“test what you can reasonably assume the learners have learned but not necessarily the same as what you have taught,” ( p.147) Obviously, this would likely achieve the course

objectives viewed from both teachers’ and students’ angles It induced teachers to cover the subject more thoroughly and motivate students to work harder to have a sense of accomplishment and thus enhance their learning However, one not-difficult-to-realize factor is that this ESP final achievement test would cause both positive and negative washback effects on teaching and leaning in terms of teaching methodology, teaching/ learning content, attitudes, behaviors, motivation, etc

II.2 Methodology of the study

II.2.1 Research approach

The research methodology applied in this study is Survey Research In order to get sufficient data for the research work, both the quantitative and qualitative methodologies were employed Specifically, in this research, questionnaire - a typical timesaving and information-reliable method of quantitative approach was exploited Besides, the main purposes of the research were also achieved with the help of qualitative method in which non-participant observation and semi-structured interviews were utilized

The reason for the choice of such methods lies in the aims to answer the two following questions:

1) What are the washback effects of the ESP end-term test on teachers’ teaching content, teaching methodology, attitudes and behaviors?

2) What are the recommendations made to teachers in order to maximize the positive washback effects and minimize the negative washback effects of the ESP end-term test for second-year students?

For each individual methodology has its own strength and weakness, one could not alone help find out satisfactory and sufficient information As a result, in order to achieve the preferable result, the researcher chose to make full use of both approaches rather than advocate one and dispose the other With such a perspective, these ways of collecting data

will “ensure validity through multiple data sources,” (Burnes, 1999, p.23)

II.2.2 Subjects

The study was conducted with the participation of 12 teachers from English for Economics Division, Faculty of English – ULIS – VNUH These teachers directly taught

Market Leader Intermediate course book to second-year students in the second semester of

academic school year 2009-2010 These are 12 female teachers with teaching experience varying from 4 years to 8 years, and experience of teaching English for General Business Purposes ranging from 2 years to 4 years Four of them obtained Master Degree in TESOL

and the rest are doing their MA course in TESOL at ULIS - VNUH

II.2.3 Methods and procedures of data collection

II.2.3.1 Methods of data collection

As mentioned above, this research was conducted based on the combination of both quantitative and qualitative methods, which include questionnaire, classroom observation and semi-structured interviews in order to get the sufficient data

 Questionnaire survey for teachers

Questionnaires have been considered “a vital tool in the collection of data ( ) it

can provide data economically and in a form that lends itself perfectly to the purposes of the study” (Gajendra and Kanka, 1999, p.117) In this particular research, the researcher

used one questionnaire set for teachers

The questionnaire for teachers consisted of 41 items (mostly close-ended), organized into four sections Section 1 with 5 questions seeks information of respondents such as age, experience in test design, etc Section 2 elicits teachers’ beliefs about factors influencing their teaching Section 3 probes teachers’ knowledge of and attitudes towards the ESP end-term test of second semester Section 4, the main part, with 21 items seeking information on different aspects of teachers’ classroom instructions regarding teaching

content, teaching methodology, teaching materials, teachers’ attitudes and behaviors, and teachers’ reactions to their students’ past test scores Detailed on teacher questionnaire can

be found in Appendix 1

 Observation technique and semi-structured interviews

Observational technique is “where the researcher stand outside of the behavior being

observed and creates a log, notes, or an audio or video record of the behavior” (Gajendra

and Kanka, 1999, p.138) In the concrete situation of this research, non-participant

observation was employed so that visibly important behaviors that questionnaire might not

reveal would be probably recognized

Three Business English lessons at UEB-VNUH were observed from late April to the middle of May (week 13, 14, 15 of the second semester) The objectives of the observation were clarified to the teacher one week earlier However, the kinds of activities that two teachers planned to have during the lesson were made unknown to the researcher before the observation began The focus of the observations was to find out whether the teachers would carry out any extra activities to prepare students for the coming ESP end-term test and students’ attitudes and reactions to such activities During observational periods, notes were taken according to observation checklist (See appendix 2 for observation checklist)

Semi-structured interviews, which often contain open-ended questions and are conducted with a fairly open framework allowing for focused, conversational, two-way communication, (Retrieved from http://wiki.answers.com/Q/What_is_semi-structured_interview) were also conducted based on the questionnaire set with two teachers who were teaching the observed classes (See appendix 3 for the interview questions) When conducting semi-structured interviews with the two teachers teaching the observed classes, the researcher would like to seek for their further remarks on why lessons were carried out that way and how influential the test was on their teaching Such extra information was so important as this type of interview could provide reliable, comparable qualitative data Moreover, when combined with the researcher’s observations, it could help the researcher cast a comprehensive look into the research issues

II.2.3.2 Procedures of data collection

The steps of data collection could be outlined as follows:

1 Studying the related materials for an overview of the research context, the background on focused issues, the research questions; and design questionnaire, observational schemes and interview questions

2 Delivering survey questionnaire to 12 teachers in English for Economics Division, Faculty of English – ULIS- VNU

3 Doing classroom observation of the two classes in the last three weeks of the semester Selection of two observed classes was done randomly with all class codes written on small pieces of papers, put in a box, shuffled and taken out by chance

4 Carrying out semi-structured interviews with two teachers teaching the observed classes The interviews were based on the questionnaire and were recorded so that they could be played for the interviewees to add extra information if desired

II.2.4 Methods of data analysis

Data analysis is “moving away from the action components of the cycle, where the

main focus is on planning and acting, to the research aspects, where the focus changes to more systematic observing and reflecting” (Burnes, 1999, p.153) As having stated before,

the data for this research was collected from questionnaire, non-participant class observation and semi-structured interviews

In analyzing the data obtained from the questionnaire, the researcher applied the data summarization methods advocated by Burnes (1999), i.e categories and graphs The data were synthesized into tables and charts with percentages calculated from the questionnaire Analysis and interpretation were then presented, summarized and finally conclusions were drawn out

Along with questionnaire analysis, the analysis of semi-structured interviews was carried out simultaneously so that the quantitative data could be compared with the qualitative data and the qualitative data, in turn, would support the quantitative data Thus, the evidence seeking for the researched questions would then be targeted The analysis was based on a set of fairly open, two-way interview questions in which both the interviewer and the interviewees flexibly exchange their ideas for the discussed issues

In dealing with the observation result, the researcher worked with the observation check list which explicitly provided the classroom data, such as teachers’ teaching content, teaching methodology, use of materials, attitudes and behaviors and students’ performance

on language as well as attitudes towards the ESP end-of-term test