Đánh giá giáo trình tiếng anh chuyên ngành dành cho học viên cao học ngành xã hội nhân văn trường đại học khoa học xã hội và nhân văn đại học quốc gia hà nội

***************** TRAN THI THU HUONG AN EVALUATION OF THE TEXTBOOK ENGLISH FOR GRADUATE STUDENTS OF SOCIAL SCIENCES AND HUMANITIES AT USSH, VNU Đánh giá giáo trình Tiếng Anh chuyên n

*****************

TRAN THI THU HUONG

AN EVALUATION OF THE TEXTBOOK ENGLISH FOR

GRADUATE STUDENTS OF SOCIAL SCIENCES AND

HUMANITIES AT USSH, VNU

(Đánh giá giáo trình Tiếng Anh chuyên ngành dành cho học viên cao học ngành xã hội nhân văn, trường Đại học Khoa học xã hội và

Nhân văn, Đại học Quốc gia Hà Nội)

M.A Combined Programme Thesis English methodology

Major code: 60 14 10

Hanoi - 2011

*****************

TRẦN THỊ THU HƯƠNG

AN EVALUATION OF THE TEXTBOOK ENGLISH FOR

GRADUATE STUDENTS OF SOCIAL SCIENCES AND

HUMANITIES AT USSH, VNU

(Đánh giá giáo trình Tiếng Anh chuyên ngành dành cho học viên cao học ngành xã hội nhân văn, trường Đại học Khoa học xã hội và

Nhân văn, Đại học Quốc gia Hà Nội)

M.A Combined Programme Thesis English methodology

Major code: 60 14 11

Supervisor: Nguyễn Thúy Hương, M.A

Hanoi - 2011

TABLE OF CONTENTS

Chapter I: Introduction 10

1.1Background to the study 10

1.2 Aims of the study 13

1.3 Research questions 13

1.4 Scope of the study 14

1.5 Design of the study 14

Chapter II: Literature review 16

2.1 Materials in language teaching and learning 16

2.1.1 Roles of teaching material EFL/ESL classes 16

2.1.2 Types of materials 18

2.2 ESP and ESP materials 20

2.2.1 A brief overview of ESP 20

2.2.2 The role of ESP materials 23

2.3 Materials evaluation 25

2.3.1 Definitions of materials evaluation 25

2.3.2 Techniques of materials evaluation 26

2.3.3 Criteria for materials evaluation 28

2.3.4 Types of materials evaluation 33

2 3 5 Models for material evaluation 34

2 4 Material adaptation 38

2.4.1 Reasons for adaptation 38

2.4.2 Techniques for adaptation 39

2.5 Previous studies on materials evaluation 41

Summary 43

Chapter III: Methodology 45

3.1 Research questions 45

3.2 Data collection instruments 45

3.2.1 Document analysis 45

3.2.2 Questionnaires 47

Summary 50

Chapter IV: RESULTS AND DISCUSSION 51

4.1 Results of data analysis 51

4.1.1 Document analysis 51

4.1.2 Data collected from questionnaires 57

4.2 Major findings 71

4.2.1 The suitability of the ESP material to the aims 71

4.2.2 The suitability of the ESP material to the content of the material 71

4.2.3 The suitability of methodology 73

CHAPTER V: CONCLUSION AND RECOMMENDATIONS 75

5.1 The strengths and weaknesses of the material 75

5.1.1 Strengths 75

5.1.2 Weaknesses 76

5.2 Recommendations 76

5.2.1 To the content 77

5.2.2 To the methodology 78

5.3 Limitations 79

5.4 Suggestion for further study 79

5.5 Conclusion 79

REFERENCES……….74

List of abbreviations

CFL College of Foreign language

VNU Vietnam national university, Hanoi

USSH University of Social Sciences and Humanities

ESP English for special purposes

ESL English as second language

ELT English Language Teaching

EFL English as a Foreign Language

TESOL Teaching English to Speakers of Other Languages

Lists of Figures and Tables

Figure 1: Materials Evaluation Model of McDonough and Shaw (1993, p.3)

Figure 2: Material Evaluation process (Hutchinson and Waters, 1993, p.98)

Figure 3: Teachers‟ experience of teaching graduate students at USSH

Figure 4: Number of ESP classes of each teacher

Figure 5: Students‟ working experience in their specialist field

Figure 6: Student‟s purposes of taking ESP course

Figure 7: Opinions of the ESP teachers and ex-students on the aims of the material Figure 8: Students‟ opinions on the difficulty level of the language items in the

Table 1 : The Selected criteria checklist for materials evaluation by

Hutchinson and Waters (1993) and Cunningsworth (1984)

Table 2 : The frequency of specialist topics students read

Table 3 : Description of language focus in the material

Table 4 : The frequency of skills covered in the material

Chapter I Introduction

1.1Background to the study

1.1.1 An overview of English teaching and learning for graduate students at USSH, VNU

The University of Social Sciences and Humanities (USSH) is a prestigious educational institution where thousands of students of social sciences and humanities at different levels, ranging from undergraduate to graduate ones are being trained They are trained to work in different fields of social sciences, including Journalism, Tourism, History, Sociology, etc After graduation, a large number of learners will need English for their jobs Therefore, English is taught as a compulsory subject, aiming at providing learners sufficient knowledge and skills During the whole course of graduate study, English accounts for 7 credits, equal to 105 class hours According to the syllabus, there are two main stages: General English (GE) and English for Specific Purposes (ESP) In stage 1 comprising the first 4 credits, students are to achieve Basic English level C with general English knowledge and communication skills 45 periods of English for Specific Purposes are expected to be covered in stage 2 However, for years, there was no ESP material for use, so during this stage, teachers and learners had

to use the last 6 units in New Headway Intermediate by Liz and John Soars (1998) instead

1.1.2 The graduate learners

To become a graduate student at the university, one has to be in the top group in entrance written exam of English and a specialized subject of their field; their results of English test scores must be from 50 out of 100 points at the minimum It means that their English should be at pre-intermediate level Thus, graduates are quite good at grammar, vocabulary and have fairly good writing skill Moreover, before taking ESP

course, the graduates have already finished the GE stage with 60 class hours and at the end of this stage they are assumed to have achieved the intermediate level of proficiency In addition, they pay much attention to ESP because they are aware of the importance of English language in general and ESP in particular to their future job and study However, most of them have very limited access to documents in English, even

in their field of work

1.1.3 The teaching staff

The Department of Foreign languages at USSH, VNU, has now become a unit under ULIS, VNU since 2009 It has a large teaching staff of English, both full-time and part-time teachers and all of them have to pass a serious recruitment exam which involves both English knowledge and skills and teaching skill, which helps to select qualified teaching staff

1.1.4 The ESP material

Graduates of social sciences and humanities are from different fields: Literature, Journalism, Psychology, International studies, Tourism, etc whose demands for terminologies and specialized English expressions are varied Practically, providing a helpful ESP program that can meet the demands of different sub-groups of learners of social sciences and humanities is a great challenge To be more precise, with one textbook, not only learners of Tourism but also other groups of students like Journalism or International studies can get useful knowledge and skills

As has been mentioned previously, for many years, the English teachers at USSH used the 6 last units in New Headway Intermediate by Liz & John Soars together with teaching materials designed or selected by themselves for the 45 periods for ESP However, the demand of a specially designed ESP material for an ESP training course of social sciences and humanities was increasing, which forced the

authorities of USSH to find a solution, and English for graduate students of social

sciences and humanities, compiled by the Dean of the Department of Foreign

language, Dr Lam Quang Dong, who is the most experienced teacher with more than

10 years teaching English at USSH, was the result This material has been officially used as the core teaching material for students of social sciences and humanities at USSH since 2008

This material contains 3 topic areas: Population and economic issues, Superstitions and customs, and Religions Each unit is organized in such sections as follows: Language focus, Reading, Listening, which in some lessons is replaced by Writing Language focus section deals with certain grammatical points in which learners are required to do rewriting exercises, gap-fill and correcting mistakes exercises or translating sentences from English into Vietnamese and vice versa In the reading section, students read a passage on some topics related to their studying fields,

like What is in a name?, Black cats and broken mirrors, Customs, and so on Learners

are provided tasks to gain some new terminologies, cultural and social knowledge and they also have much chance to practice reading and translating skills For Listening, teachers in charge of each class have to select their own listening passages to teach their students In the writing section, the author provides different kinds of exercises varying from gap-fill to matching, writing stories or essays and describing graphs or charts, etc in order for students to build sentences and extend them into paragraphs and essays The material was highly appreciated by the administrators, teachers and

students at USSH because it seemed to meet learners‟ needs

1.1.5 Arising problem

However, after two semesters using the new course-book, there have been a lot

of reflections from both teachers and learners on the appropriateness of the material and the learning outcomes Some reflections were positive but some were extremely negative Therefore, an investigation into the appropriateness of this textbook to

learners‟ needs would be of great importance, as this study would provide information for the author to improve it so that students‟ language levels and teaching aims are met

These above mentioned facts inspired me to conduct this study “An evaluation

of English for graduate students of social sciences and humanities at USSH, VNU”

Hopefully, the results of this study will be of much help to the teachers of graduate students at USSH in achieving the objectives of the ESP program in the coming years

1.2 Aims of the study

The paper is aimed at evaluating the textbook English for graduate students of

social sciences and humanities in terms of aims, content, methodology based on users‟

responses toward its appropriateness to the course requirements Based on the findings, recommendations on how to improve it, and thereby, improve the effectiveness of ESP teaching and learning at USSH shall be given

And the specific aims are:

 to evaluate the appropriateness of the material to the course requirements in terms of aims, content, and methodology Then areas for improvement of this textbook will be identified

 finally, to raise some recommendations to improve this textbook in order to meet the learning demands of learners

 In addition, theoretically the improved textbook of social sciences and humanities, which is rather rare, can be an effective and helpful referential material for teachers and learners

1.3 Research question

To achieve the above mentioned aims, the following research question was posed:

- To what extent does the ESP material English for graduate students of social

sciences and humanities at USSH, VNU meet the course requirements in

terms of aims, content, and methodology?

1.4 Scope of the study

Hutchinson and Waters (1993), Brown (1995) and many other language evaluation experts agreed that evaluating a teaching material is a complicated process based on different criteria such as audience, content, physical appearance, cultural bias, needs and so on However, because of time limitation, the researcher just focuses on the criteria stated in the requirements of the ESP course for graduates of social sciences and humanities at USSH, VNU, including aims, content and methodology and time allocation

1.5 Design of the study

The paper is organized into five chapters:

1 Chapter I is the introduction that presents such basic information as rationale,

aims, methods, research questions, and design of the study

2 Chapter II presents a review of related literature that provides introduction to

textbook and textbook evaluation but the focus is on the importance of the appropriateness of the textbook and learning targets Some understandings about criteria of a good textbook will also be proposed

3 Chapter III consists of rationale for the research method applied, description of

three groups of subjects, data elicitation tools and data collection procedures

4 Chapter IV, the main section of the study, presents the data analysis and

findings concerning students‟ language levels in an ESP course, the present

situation of using English for graduate students of social sciences and

humanities at USSH in order to lay bare its strengths and weaknesses Based on

these, an evaluation of the appropriateness of this textbook to learning targets was done

5 Chapter V proposes some recommendations from the perspectives of teaching

staff and students in terms on how to improve this textbook so that learners‟ needs were met In addition, the limitations of this study are also presented

Chapter II Literature review

2.1 Materials in language teaching and learning

2.1.1 Roles of teaching material EFL/ESL classes

The relationship between materials and other components in language teaching such as students, teachers, teaching method and evaluation have been controversial for many years Since the end of 1970s, there has been a movement to make learners rather than teachers the center of language learning According to this teaching approach, learners become the targets of teaching and learning process Therefore, materials, curriculum, teaching method and evaluation should be designed for learners and their needs For the teaching and learning English to be successful, besides teachers, learners, curriculum, method and evaluation, materials play a decisive part in language teaching and learning The importance of teaching and learning materials can not be denied and so far, there have been a great number of discussions on different perspectives on the role of materials in relation to other elements Teaching materials are considered the essential components in most language courses because they help students and teachers know teaching and learning content Discussing the importance

of English language teaching materials to learners, Allwright (1981) indicates that materials help students know what to learn In addition, he says, they are resource for ideas, activities for instruction and they give teachers rationale for what to do He also emphasizes that materials control learning and teaching It is undoubted that materials are of importance in teachers‟ language instruction Both experienced and inexperienced teachers rely on the materials O‟Neill (1982) supports Wright‟s ideas that materials help learning and teaching It is true that in many cases, teachers and students rely heavily on materials and materials determine the components and

methods of learning They control the content, method and procedures of learning Students learn what is presented in the textbook and the way the textbook presents materials in the way the teacher teaches it Therefore, the materials become the center

of instruction and one of the most important influences on what goes on in the class In this view, Tomlinson (1998, p.20) confirms “materials facilitate then curriculum, teaching method and evaluation in language learning, increase the learners‟ knowledge and experience of the language” He considers materials as the basic for much of the language input that the learners receive and as the sources for much of the language practice that occurs in the classroom

Dudley-Evans and St John (1998) also indicate the four functions of teaching materials The first function is that materials are authentic source of language, which means that materials need to present real language as it is used in real situation and be suitable to a wide range of learners‟ needs The second one is that materials are a source of learning support To enhance learning, according to the writers, the materials need involve students in thinking about and using language; the activities in the materials need to “stimulate cognitive, not mechanical process”; and the learners also need “a sense of progression” (p.171) The third one is that materials are a source for stimulation and motivation To fulfill this function, materials need to be “challenging yet achievable; to offer new ideas and information whilst being grounded in the learner‟s experience and knowledge” (Dudley-Evans and St John, 1998, p.172) These authors also explain that the input must contain concept and knowledge that are familiar to the learners, but it must also offer something new, a reason for learners to communicate and to get involved The last one is that materials are a source of reference, which is understood as that materials need to be “complete, well-laid out and self-explanatory” (Dudley-Evans and St John, 1998, p.172) In other words, materials

should provide explanations; examples and practice activities that have answers and

discussion keys do that learners can use them for their self-study

Obviously, good teaching materials are of great help to both inexperienced and experienced teachers, especially the inexperienced or poorly trained teachers because they also provide models for teachers to follow in developing their own materials (Nunan, 1998) Therefore, the first thing the teachers need to do when preparing his course in his own situation is to write or to select the appropriate materials

In USSH, English for post graduates of social sciences and humanities was

designed by Lam Quang Dong, Dean of the Department of foreign language Due to the lack of time and the unavailability of various resources, the material was compiled from various texts and most of the exercises were designed by the author After being used for two years, it is necessary to evaluate the material to see if it were appropriate

or not

2.1.2 Types of materials

The following section is devoted to a brief account of different types of materials and the advantages and disadvantages of using these types of materials According to Robinson (1991), the first type is published textbooks and the second type is in-house materials

 Textbooks

The term “textbook” refers to course book, of which the teacher and usually each student has a copy, is principle to be followed systematically as the basic for a language course There are some arguments for and against using a textbook According to Ur (1996), textbooks have many advantages as follows:

A textbook provide a clear framework: teachers and learners know where they are going and what is coming next In many places, the textbook serves as a syllabus Besides, the textbook provides texts and learning tasks, which are likely to be of an

appropriate level for most of the students It is also the cheapest way of providing learning material for each learner (p.184)

However, not everything in the textbook is wonderful (Hammer, 1991) According

to Hammer and Ur, though the textbooks are well planned, they can be inappropriate for teachers and learners The counter-arguments for using textbooks are also listed by

on the part of the learners

- Homogeneity: textbooks have their own rationale, chosen teaching, and learning approach They do not usually cater for the variety of levels of ability and knowledge, or learning styles and strategies that exist in most classes

- Over-easiness: teachers find it too easy to follow the textbook uncritically instead of using their initiative; they may find themselves functioning merely as mediators of its content instead of as teachers in their own right

Therefore, once using a textbook, teachers must consider these counter-arguments

 In-house materials

In-house materials or institutionally prepared materials are also the teaching materials that teachers use (Richards, 2001) One of the most importance factors that in-house materials have is that they are always designed for particular group of learners

so they correspond to learners‟ needs and match the aims and objectives of the course

According to Robinson (1991), using in-house materials, has many advantages as follows:

“First, in-house materials are likely to be more specific and appropriate than published materials and to have greater face validity in terms of the language dealt with and the contexts it is presented in second, in-house materials may be more flexible than published textbooks Last, the writer of in-house materials can make sure of the suitability of methodology for the intended learners However, in-house materials take time to write and their production is relatively expensive” (p.58)

In short, both textbooks and in-house materials have their own advantages and disadvantages Which one should be used depends on the purposes of the course as well as particular situation Which category does the material under investigation fall

into? In this particular situation, English for graduate students of social sciences and

humanities belongs to in-house material section

2.2 ESP and ESP materials

2.2.1 A brief overview of ESP

English for Specific purposes (ESP) has become an important part of English language learning People want to learn English, not for the pleasure or prestige of knowing the language, but because English is the key to the international currencies of technology and commerce Knowledge of a foreign language has been generally regarded as a sign of a broad education English created a new generation of learners who knew specifically why they were learning a language such as doctors who needed

to keep up with developments in their field, etc

From the early 1960‟s, the development of English for Specific Purposes has been reflected in the increasing number of ESP courses offered for overseas students in English speaking countries Since then, the teaching of English for Specific Purposes (ESP) has been a lively stimulating part of English language teaching (ELT) It has

been generally acknowledged that, while the remaining a part of ELT, ESP developed its own procedures, such as needs analysis, its own materials and its own teaching methodology (Dudley-Evans and St John, 1998) The emphasis in the definition of ESP has been on how ESP teaching develops procedures appropriate to learners whose main purpose is learning English for a specific aim/use other than just learning the language system That purpose may be educational, or may be professional, and ESP seeks its justification on how well it prepares learners to fulfill the purposes required of them ESP has been defined by many researchers and writers Anthony (1997) notes that there has been considerable recent debate about what ESP means despite the fact that it is an approach that has been widely used over the last four decades Mackay and Mountford (1978) indicated that the term ESP is generally used to refer to the teaching

of English for a clear utilitarian purpose El-Minyawi (1984) pinpointed that ESP courses are based solidly upon the need to express the facts and ideas of some special subjects after which the student should be able to read the specialized subjects confidently and speak about them fluently The definition of Strevens (1988) makes a distinction between four absolute characteristics and two variable characteristics including:

Absolute characteristics:

- ESP is designed to meet specific needs of the learners

- ESP is related in content (that is in its theme and topic) to particular disciplines, occupation and activities

- ESP is centered on language appropriate to those activities in syntax, lexis discourse

- ESP in contrast with “General English”

Variable characteristics:

- ESP may be restricted to the learning skills to be learned

- ESP may not be taught according to any pre-ordained methodology

At the 1997 Japanese Conference on ESP, Dudley-Evans offered a modified definition The revised definition by Dudley-Evans and St John postulates with absolute and variable characteristics as follow:

+ Absolute characteristics:

- ESP is defined to meet specific needs of the learner

- ESP makes use of the underlying methodology and activities of the discipline it serves

- ESP is centered on the language (grammar, lexis), skills, discourse and genres appropriate to these activities

+ Variable characteristics

- ESP may be related to or designed for specific disciplines

- ESP may use, in specific teaching situations, a different methodology from that

of general English

- ESP if likely to be designed for adult learners, either at a tertiary level institution

or in a professional work situation, It could, however, be for learners at secondary school level

- ESP is generally designed for intermediate or advanced students

- Most ESP courses assume some basic knowledge of the language system but it can be used with beginners (Dudley Evans & St John, 1998, p.4-5)

In my point of view, the best definition for ESP is the one by Hutchison and Waters (1987), they defined it as an approach to language teaching in which all decisions as to content and method are based on the learner‟s reason for learning They added also that the term ESP implies that it is English which is somehow peculiar to the range of principles and procedures that define that particular profession Besides, ESP does not involve a particular kind of language, teaching material or methodology

Regarding types of ESP, Carter (1983) identifies three types of ESP:

- English as a restricted language (for example, the language used by air traffic controllers or by waiters)

- English for Academic and occupational purposes

- English with specific topics

The language used by air traffic controllers or by waiters are examples of English as a restricted language

The second type of ESP identified by Carter (1983) is English for Academic and Occupational Purposes In the “Tree of ELT” (Hutchison and Waters, 1987), ESP is broken down into three branches: English for Science and Technology (EST), English for Business and Economics (EBE), and English for Social Studies (ESS) Each of these subject areas is further divided into two branches: English for Academic Purposes (EAP) and English for Occupational Purposes (EOP) An example of EOP for the ESS branch is “English for Social Sciences and Humanities” In other words, the

studied material English for graduate students of social sciences and humanities is an

EOP branch of ESS

The third and final type of ESP identified by Carter (1983) is English with specific topics Carter notes that it is only here where emphasis shifts from purpose to topic This type of ESP is uniquely concerned with anticipated future English needs of, for example, scientists requiring English for postgraduate reading studies, attending conferences or working in foreign institutions

In short, it is clear that ESP plays very important role in the English language teaching and learning Therefore, it is extremely necessary to pay much attention to the teaching and learning of ESP

2.2.2 The role of ESP materials

It is undeniable that materials are a key component in most language courses and teaching materials is vital in the process of successful language teaching and learning Materials are also the center of instruction and one of the most important

influences on what goes on in the classroom Many authors indicate this crucial role that teaching materials play in language teaching Brown (1995, p.139) defines

“materials as any systematic description of the techniques and exercises to be used in the classroom teaching” Allwright (1981) also accepts that materials should teach students to learn, that they should be resource books for ideas and activities for instruction/learning, and that they should give teachers rationales for what they do Similarly, according to Cunningsworth (1995), teaching materials are sources of ideas and stimulation for classroom activities, which provide learners with practice in communicative interaction Besides, teaching materials can function as a reference source for learners on grammar, vocabulary and pronunciation Moreover, teaching materials can be considered as a syllabus reflecting learning objectives that have already been determined and a support for less experienced teachers who have yet to gain confidence At this point, materials are vital resources because they stimulate and develop students‟ linguistics skills This role is even more important and essential in the ESP context as in this point, ESP materials can be the primary source of exposing learners to the language they need to learn

In short, teaching materials have great efforts on the success of any language teaching program, especially the teaching and learning of ESP The teacher needs to take into account the roles as well as the requirements of teaching materials when designing, selecting or changing material for his/her own teaching situation to make them suitable to the learners and the aims of the course In the particular situation at USSH, VNU, ESP material of social sciences and humanities plays a key role in providing both teachers and learners sources in their teaching and learning process Without it, they will get a large number of difficulties in finding materials, identifying what to teach and learn and they likely have to spend a lot of time and efforts during their ESP course

2.3 Materials evaluation

2.3.1 Definitions of materials evaluation

Until now, there have been a number of materials evaluations of different author

in ELT According to Tomlinson (1998, p.3), materials evaluation is defined as “a systematic appraisal of the value of materials in relation to their objectives and to the objectives of learners using them.” (Ellis, quoted in Figen, 2002, available at http://www.metu.edu.tr./home/wwwsbe/thabs/2002/EDS.htm)

Hutchinson and Waters (1993, p96) also give definition of materials evaluation,

as “Evaluation is a matter of judging the fitness of something for a particular purposes Given a certain need, and in light of the resources available, which is out of a number

of possibilities can represent the best solution?” In other words “Evaluation is basically

a matching process: matching needs to available solutions.” Hutchison and Waters

(1993, p.96) also give definition of material evaluation as “Evaluation is a matter of judging the fitness of something for a particular purposes Given a certain need, and in the light of the resources available, which is out of a number of possibilities can represent the best solution?” In other words “Evaluation is basically a matching process: matching needs to available solutions.” (Hutchinson and Waters (1993, p 97) Dubin and Olshtain (1986, p.3) support Hutchinson and Waters by stating “evaluation means the same as testing and that while students are being tested evaluation is taking place in terms of judging the fitness of something for a particular purposes.”

In conclusion, from the general definition of evaluation and the definition of specific area of materials evaluation, it can be inferred that materials evaluation involve the determination of what needs to be evaluated, the objectives and requirements of the materials, and the judgments of the value of the materials being evaluated in relation to the objective and requirements determined

2.3.2 Techniques of materials evaluation

As mentioned in section 2.3.4, there are three types of evaluation and each of them has their own method to collect data (Robinson, 1991) The importance is that researchers should identify the suitable methods in the study A lot of technique used to evaluate materials attract much attention and have been discussed by many researchers: Dudley-Evan and St John (1998) According Hutchinson & Waters (1993), Lynch (1996), Richards (2001), Robinson (1991), and according to Hutchinson & Water (1993, p.153), the useful evaluation techniques area questionnaires, interviews, test results, discussion and other informal means

Dudley-Evan & St John (1998, p.132) stated that in evaluating and ESP course, one or more of the data collection methods can be used: checklist and questionnaires, accession, discussion and record keeping

The different methods used for evaluation have been mentioned but they all have their possible advantages and limitations as follows

Questionnaires are the most popular methods of data collection They can be used to elicit students‟ and teachers‟ comments on a wide range of issues The advantage is that they can be given to a large group of respondents at the same time and they are not expensive (Wallace, 1998) However, questionnaires need to be carefully designed to avoid respondents‟ misinterpretation of questions

Interviews, as Robinson (1991) stated, are also a valuable method for evaluation They have advantages of in-depth information on specific questions but the problem with this type of instruments is that it is very time-consuming though they can provide the opportunity for more extended exploration of the issues than questionnaires

Observation can focus on any observable aspect of the lesson and can provide

an objective eye to identify things that may not be very clear to the teacher However,

it is a specialized work and requires careful preparation

Document analysis refers to any documents that are involved in evocation, such as: curriculum description, policy statement, newspapers, and program brochures, correspondence and so on (Lynch, 1986) According to Robinson (1991), document analysis can be records He points out that records can be reports of lessons taught, statements of needs, aims and objectives of the course It is also trencher‟s checklist used to evaluate the lesson taught In brief, according to him, records can be any documents that are related to the course valuation

The numerous authors have discussed some other types of instruments used (including checklists, discussion and tests) to collect data However, each stage requires suitable instruments Robinson (1991) suggests that questionnaires, interviews and tests should be used for formative evaluation but summative evaluation needs the use of questionnaires, checklists and tests

To sum up there has been a large number of different methods to collect data The researcher should identify what types of these instruments to gee used effectively

in their study For example, with the aims of evaluating the effectiveness of the textbook used for her students Vo (2004) used document analysis, interviews and questionnaires to collect data Nguyen (2004) applied the materials analysis in addition

to questionnaires to gather information from both teachers and students They all have common techniques, using questionnaires to collect data because of the big numbers of participants in their research Also, from their studies, questionnaires were suggested to

be the best method for date collection in their particular teaching contexts

According to Hutchinson & Water (1993) and Robinson (1991), employing instruments in an evaluation project will depend on what suits the particular situation;

the combination of two methods including document analysis and questionnaires is applied for data collection in this study The reason is that the researcher realized that these techniques are suitable for the evaluative purpose of research and have a lot of advantages, which would be discussed in Chapter III

2.3.3 Criteria for materials evaluation

Criteria for materials evaluation used to determine which perspectives of the material should be taken into consideration Tomlinson (1998, p.220) suggests,

“Criteria are what researchers use to reach a decision regarding what needs to be evaluated Defining criteria become one of the major steps of the evaluation process that researchers must take into account before any evaluation take place Until now, different sets of criteria have been developed to make the task of evaluating material more efficient and accurate In this study, two sets of criteria, which are popular in materials evaluation, will be reviewed The first one is suggested by William (1983) and Hutchinson and Waters (1993) suggest the second one

* Criteria defined by William

William (1983) suggests a wide range of criteria that can be used to evaluate almost all aspects of materials The set of criteria proposes by William include linguistic and pedagogical criteria, general criteria and technical criteria Each of the criteria has the following aspects:

Linguistic and pedagogical criteria: the completeness and appropriateness of the items presented, activities suggested for practicing the items selected, sequencing of vocabulary and functional load, relevance of its contexts and situations

Gerald criteria: Global considerations of methodology, needs of the learners, teachers and community

Technical criteria: quality of editing and polishing Availability of supplementary, materials, cost and durability of text and authenticity of language and style of writers (William (1983, p.252))

The criteria suggested by William seem to be very useful and appropriate to evaluate a course book However, such frameworks are more suitable for the selection of course books available in the market than for the examining of an in-house material to see whether they meet the intended objectives The author of this study would like to present other sets of evaluating criteria, which can be useful and help the evaluation in this study

* Criteria defined by Hutchinson and Waters

Hutchinson and Waters (1993) present a group of criteria, which includes audience, aims, content and methodology

Audience

According to Hutchinson and Waters, looking at the audience of the materials means checking what kind of learners the material is intended for Though the in-house material designed particularly for a defined group of learners, it is still necessary to see

if the material designer took into consideration all the information of the learners or not So the first criterion in materials evaluation is the audience of the materials with regards to different information such as: ages, sex, study or professional fields, status, knowledge of English, of areas of work or study, educational background, interest and

so on

Aims

The aims of the material is to help learners develop certain language skills and function or to improve the language as a whole In any language course, it is necessary

to examine whether the material can satisfy the aims and the objectives of the course or not

Content

The content is one of the important elements of the material In order to measure the material‟s content to see if it is suitable for the course, the researchers need to take into account the following items: language description, language points, macro-skills and their proportion, micro-skills, text – types, subject matter areas, topics, organization through the course, organization within the course units, content sequence, content sequence within a unit It is necessary to check if these items in the material satisfy the requirements, which were set up from the analysis of the learners‟ needs and the course‟s objectives

Methodology

Other criteria given out by Hutchinson and Waters are methodology The criterion refers to the researcher‟s judgment of whether the methodology used in the material is appropriate with the learners (especially the learners‟ preference) and efficient in helping them achieve the course‟s objectives Methodology of the material can be reflected by the theories of learning on which the course is based, the attitudes

to or the expectations of the learners about learning English, kinds of exercises and tasks included, teaching and learning techniques used, teaching aids, teaching guidance needed and the flexibility of the material to suit in different teaching contexts

Besides the four criteria mentioned above, Hutchinson and Waters suggest other criteria such as the price and availability of the material

Comparing the two sets of criteria presented above by William (1983) and by Hutchinson and Waters (1993), it is obvious that the work of developing and choosing evaluative criteria is rather subjective and depends on what the researchers consider to

be important In this evaluation research, the four criteria including audience, aims, content and methodology suggested by Hutchinson and Waters (1993) have been

selected Although English for graduate students of social sciences and humanities is

an in-house material designed particularly for an intended group of learners, it is necessary for the researcher to check if the four main criteria above were taken into consideration Besides, it appears possible that the four criteria presented above could

be examined for the material evaluation in the context at USSH, VNU

Although the four criteria suggested by Hutchinson and Water (1993) consist of many issues, this study just concentrated on the following ones:

Audience

1A Who are your learners?

Age Sex Study of work specialist

Knowledge of English specializes

3A What language points should be

covered? (What particular grammar,

structures, vocabulary areas?)

3B What language points do the materials cover?

4A What macro – skills are taught? 4B What macro-skills are covered in the

materials?

5A What proportion of work on each

macro-skill? Is there integrated work? 5B What proportion of work on each macro-skill? Is there integrated work? 6A What micro-skills does the course

- What level of knowledge should

be assumed (e.g second year/ third year

college)?

- What types of topics are needed?

8B What are the subject matters assumed level of knowledge, and types of topic in the materials?

Methodology

9A What kinds of tasks and exercises are

needed? 9B What kinds of tasks and exercises are included in the materials? 10A What teaching – learning techniques

are to get used? 10B What teaching-learning techniques can be used with the materials?

Lockstep Pair work Group work Students presentation Other kinds 11A What aids are available for used?

12A What guidance/ support for teaching

the course will be needed?

12B What guidance do the materials provide?

Table 1: The Selected criteria checklist for materials evaluation by Hutchinson and Waters (1993) and Cunningsworth (1984)

2.3.4 Types of materials evaluation

Until now, there have been different terms to indicate types of materials Ellis (1997) introduces two terms of materials evaluation including predictive and retrospective The first one is used to get the decision regarding which material to use while the second one, the retrospective evaluation, is used to examine whether the materials have been used effectively or not

Besides these terms, Robinson (1991) classified materials evaluation into three types, which are very clear to understand: preliminary, formative and summative evaluation

Preliminary evaluation often takes place before the course starts This type of

evaluation is used to select the most appropriate materials from the publications that will be suitable with the group of learners and the aims of the course “This also has the purpose of identifying which aspects of the published materials need to be adapted to suit the purpose of researchers” (Ellis, 1997, cited in Tomlinson, 1998, p.220)

Formative evaluation normally takes place during the development of the

program and its curriculum and the findings will be used to improve program

Summative evaluation is normally carried out at the end of the course or some

time after the course has been implemented Johnson (1989, p.229) says: “the purpose

of summative evaluation is to determine whether the program was successful and effective” According to Ellis (1997), summative evaluation is very important after each course because the findings of this evaluation will lead to the decision whether to repeat or use the materials again or not Other problems can be solved basing on the findings of the evaluation as well Answers for the questions which tasks and activities are suitable and which are not; which aspects of the materials need to be changed or how to modify them to make more effective and efficient for future use can only be achieved after the careful evaluation of the materials In short, summative evaluation is used to evaluate the worth and value of teaching materials after they have been used

In short, there are different types of materials evaluation; so when making an evaluation project the researchers have to determine what type of materials evaluation should be adapted in their work For USSH situation, the material under evaluation has been selected and have been used for two years, therefore, the researcher used summative evaluation to conduct a careful evaluation and some recommendations were made to improve the effectiveness of the materials

2 3 5 Models for material evaluation

Many famous language researchers, language teachers and course planners have suggested models for materials evaluation The most popular ones, however, were suggested by Hutchinson and Waters (1993), Cunningsworth (1984) and McDonough and Shaw (1993)

 Evaluation model by Cunningsworth (1984)

Cunningsworth (1984) believes that the process for material evaluations should

follow four basic principles:

The first one refers to the teaching materials, teaching aims and objectives It means that the materials or the textbook chosen should be suitable with the goals and objectives of the course in a specific situation

The second principle is that the selection of the materials should be based on students‟ language levels and fulfills their needs Moreover, they should be practical Following this principle, the selected teaching materials help equip the students to use language effectively for their own purposes They also make teachers look beyond the confines of the classroom into the outside world

The third one is to keep students‟ learning needs in mind Their learning needs are understood not only as the actual language to be learned as it is selected, graded, presented, and practiced but as also the small pieces of new language that students may come across at each stage

The last one is the combination of the relationship among the language, the learning process and the learners The detailed steps involved in the evaluation model suggested

by Cunningsworth (1984) are:

1 Briefly state the objectives of the material

2 To what extent is it successful in achieving these objectives?

3 Note particular strengths

4 Note particular weaknesses

5 Are these any notable omissions?

6 For what types of learning situations is the material unsuitable?

7 Comparisons with any other material evaluated

Evaluation model by McDonough and Shaw (1993) is a combination of and Micro evaluation In this model, they suggest a three-stage evaluation model called: external evaluation, internal evaluation and overall evaluation

Macro-The external stage (Macro evaluation) comprises the claims made for the

materials by the author/publisher with intention to: the intended audience, the proficiency level, the context and the presentation of language items All of these things can be found on the cover of the book, the instruction of the materials and the table of contents The aim of this evaluation stage is to identify whether the material is potentially appropriate, then the internal stage starts

The internal stage (Micro- evaluation) is used to examine the extent to which

the factor in the external evaluation stage actually matches the internal consistency and organization of the materials staged by the author/publisher This stage requires

an in-depth investigation into the materials so we need to examine at least two or more units of the book

The final step is the overall evaluation This stage helps to assess the suitability

of the materials for specified groups or individuals by considering some of the factors such as the usability, the generalizability, the adaptability and the flexibility This model is shown in figure 1

Macro-evaluation  inappropriate/ potentially appropriate

(External)

EXIT Micro-evaluation  inappropriate/ appropriate  adopt/ select

(Internal)

EXIT

Figure 1: Materials Evaluation Model of McDonough and Shaw (1993, p.3)

 Evaluation model by Hutchinson and Waters (1993)

Evaluation by Hutchinson and Waters (1993) is a perfect Macro-Evaluation This model seems to be clear and useful when he divided the material evaluation process into four mains steps The first one is to define the criteria which the evaluation is based on The second one is to determine the subjective analysis The third one is to

determine the objective analysis and the last one is to compare the findings with the material requirement to evaluate the match between the material and the requirements

The whole set of material evaluation process is presented in the following diagram

Figure 2: Material Evaluation process (Hutchinson and Waters, 1993, p.98)

As can be seen from Figure 2, this model presents a logical procedure for material evaluation This first step is defining criteria In this step, the researcher has to define what bases the material will be judged on and which criteria the researcher wants to include in the course The second step is to identify the requirements for material so the researcher should describe in detail on what criteria your course is based The third one

is to evaluate whether the existing material realize the criteria set in the subjective analysis The final step is the matching process, which finds out how far the material matches the course requirements

In summary, Hutchinson and Waters (1993) suggest the model for material evaluation that presents a logical procedure This model also has a clear-cut description

of the stages and criteria, which are presented systematically Besides, it can help the

Define criteria

- On what bases will you judge materials?

- Which criteria will be more important?

Matching

How far does the material match your needs?

Subjective analysis

- What realization of the criteria

do you want in your course?

Objective analysis

- How does the material being evaluated realize the criteria?

researcher knows exactly what must be done to analyze the material in comparison with the course requirements Therefore, in order to determine whether the existing material is suitable for graduate students of social sciences and humanities at USSH, VNU, the evaluation framework by Hutchinson and Waters (1993) has been chosen It

is very important for the author to set out the criteria for both subjective analysis (i.e the analysis of the training curriculum) and objective analysis (the analysis of the material being evaluated) Then the findings of the two analyses are put into comparison to find out whether they match and if not, to what extents they do not match up Based on the results, good aspects as well as problematic parts of the material can be identified, which may lead to some suggestions for the material improvements

2 4 Material adaptation

2.4.1 Reasons for adaptation

According to Mc Donough and Shaw (1993, p.85) “Materials adaptation is a process of matching and its purposes is to maximize the appropriateness of teaching materials in context, by changing some of the internal characteristics of a course book

to better suit our particular circumstances.” He also gives the reasons for modification and some changes that you wish to make to your own materials These reasons for adaptation which rely on the whole range of variables in your own teaching situation are listed by Mc Donough and Shaw (1993, p.87) as follows:

 Not enough grammar coverage in general

 Not enough practice of grammar points of particular difficulty to these learners

 The communicative focus means that grammar is presented unsystematically

 Reading passages contain too much unknown vocabulary

 Comprehension question are too easy

 Listening passages are inauthentic

 Not enough guidance on pronunciation

 Subject-matter inappropriate for learners of this age and intellectual level

 Photographs and other illustrative materials are not acceptable

 Amount of material too great or too little to cover in the time allocated to lessons

 No guidance for teachers on handing group work and role-play activities with a large class

 Dialogues too formal, and not really representative of everyday speech

 Audio material difficult to use because of problems to do with room size and technical equipment

 Too much or too little variety in the activities

 Vocabulary list and a key to the exercises would be helpful

 Accompanying test needed

Apart from the reasons mentioned above, Mc Donough and Shaw (1993) state that much more can be added to the list but it must depend on each teaching context or course requirement Consequently, all aspects of the language classroom covered can

be divided into four aspects: “language use; skills; classroom organization; supplementary material.” (p.86)

2.4.2 Techniques for adaptation

According to Mc Donough and Shaw (1993), the procedures that can be applied

in order to bring about changes are the techniques These individual techniques can be discussed and presented as follows:

Adding

The notion of addition can be implied that materials are supplemented by putting more into them quantitatively and qualitatively In the quantitative way, we can add the technique of extending and a more far-reaching perspective on addition of material “Techniques are being applied within the methodological framework of the

original materials: in other words, the model is not itself changed” (McDonough and Shaw, 1993, p.89) The qualitative way refers to the development of the methodology

in new directions by putting in different language skill or a new component Therefore, addition can be made before, during or after the intended area For instance, teachers may let students do in advanced some supplementary exercises before they begin their practice with new and quite difficult grammatical points in the material

Deleting or omitting

In contrast with addition, the process of deletion needs no further clarification This means that things deemed inappropriate, offensive, unproductive for a particular group must be left out Similar to addition, the technique of deletion can be used over a part of an exercise or of a whole unit of a course book Therefore, addition and deletion often work together The task in the material may be deleted and compensated by a more suitable one to meet the objectives of the whole unit However, when the techniques for adaptation are applied, it is necessary to take into account the balance of the lesson as well as the time allocation

Modifying

Modifying can be understood as an internal change in the approach or focus of

an exercise or other piece of material Modification can be divided into two specific ways: re-writing and re-structuring Re-writing refers to the modification of the linguistic contents whereas the re-structuring applies to the classroom management, especially the structuring of the class In short, modifying is a technique that has a very wide range of application It can be a change of a small component in the material such

as an exercise or text or a classroom activity

Simplifying

The technique of simplifying is similar to one type of the technique of modifying, namely re-writing activity Many elements of the material can be simplified but the main application of this technique has mainly applied to the texts or reading

passages We can simplify the texts with sentence structures, the grammar structures and the lexical content to be closer to the proficiency level of a particular group of learners

Re-ordering

The technique of re-ordering refers to the possibility of putting parts of a course book in different order, adjusting the sequence of presentation within a unit, or arranging of different units in a course book

Re-placing

The materials after being evaluated may have texts or exercises that are ineffective or inappropriate to the objectives of the course, ones that are more suitable may replace these

Adaptation is a very practical activity which plays an important part in improving the effectiveness of the materials for teachers‟ and learners‟ usage It makes the teaching materials more relevant and appropriate for a particular group of learners Therefore, every teacher should take the technique to be selected depends on the teaching context and students‟ learning activity In the context of USSH, the techniques

of adding, modifying, deleting or omitting would be used to help make the material under evaluation more effective and suitable to the teaching context here

2.5 Previous studies on materials evaluation

Evaluation is obviously a world-wide activity which has been more and more appreciated by teachers and researchers According to Brown (1989, p223), evaluation

is the systematic collection and analysis of all relevant information necessary to promote the improvement of a curriculum and assess its effectiveness and efficiency

Up to now, there have been many studies done in the field of evaluation of ESP materials Many of these studies have brought about good improvement for the textbooks or teaching materials themselves and for ESP courses We can count on Figen (2002), Griffiths (1995), James (2002) or Nguyen Thi Thai Ha (2004,), Vo Thi

Anh Dao (2004), Tran Thi Thuy Nga (2005), Dong Thi Hien (2006) or Hoang Thi Tham (2007) These evaluation had been done on different kinds of materials such as ESP for engineering students, for Electronics and Telecommunications students, for marine students, for information technology students or medical students Vo Thi Anh Dao (2004) made an evaluation on the ESP textbook used for the students of Husbandry and Veterinary Science in 2000 In her study, Nguyen Thi Thai Ha (2004) evaluated the ESP material used for freshman postgraduate students at Hanoi University of Technology in 1999 Vu Thi Lua (2005) had a critical look at the ESP course book for students of Business and Management at Hanoi UBM and then she made some recommendations for a better use Dong Thi Hien (2006) made an evaluation of ESP materials used for the second year students of Security at People‟s Security Academy Hoang Thi Tham (2007) evaluated the reading materials for students of mechanical engineering at Thai Nguyen university of Technology All of these researchers of ESP materials aimed at finding out the strengths and the weaknesses of the materials and then usually gave suggestions for improving the materials As this thesis aims at evaluating the ESP material for postgraduate students

of social science and humanities in terms of language level, aims, content and methodology, to determine whether it meets students‟ language levels and the course‟s objectives, the researcher just want to review two Vietnamese studies that have the similar aspects shall be reviewed in more detail

Pham Thi Thu Huong (2006) carried out an evaluation of the ESP material

“English in medicine” for 3rd year students of medicine at the Military Academy of Medicine (MAM) in terms of aims, content, methodology and to determine whether the material is suitable to the students with regards to their level of English The subjects

of this study were 85 fourth year students of MAM who were taught the material in the fifth semester and five English teachers who had taught the material for more than 5 years The study employed a combination of two data collection instruments, namely