An evaluation of the textbook tiếng anh 6 (10 year program a case study at quynh mai secondary school

However, in this study, due to time constraints and scope of the minor thesis, the main focus of the evaluation is on the following criteria as audience, aims of the material, content of

COVER PAGE 2

VIETNAM ACADEMY OF SOCIAL SCIENCES

GRADUATE ACADEMY OF SOCIAL SCIENCES

VIETNAM ACADEMY OF SOCIAL SCIENCES

GRADUATE ACADEMY OF SOCIAL SCIENCES

DECLARATION BY AUTHOR

I declare that this thesis entitled “An Evaluation of the Textbook

“Tiếng Anh 6 (10-year Program)”: A Case Study at Quynh Mai Secondary School” is created by my own findings and I am the sole author of this thesis

To the best of my knowledge, with the exception of the indication of reference, this thesis is carried out without using any other author’s work and

is submitted after a carefully checking progress in order to fulfill the requirements of the M.A degree

Author’s signature

Nguyễn Thị Viên

Approved by SUPERVISOR

Phạm Lan Anh, Ph.D

Date:

ACKNOWLEDGEMENTS

This thesis is completed with lots of working hours and is contributed

by a large amount of enthusiasm, effort and assistance from many people

First and foremost, I would like to express my gratitude to my supervisor, Dr Phạm Lan Anh for her support Without her guidance, advice and comments, I would not complete this research paper

Moreover, I would like to send my special thanks to all of the teachers

at Quynh Mai Secondary School for their help and suggestions which are very necessary and useful to me during the progress of this thesis

Additionally, I am so grateful for the assistance from the students at Quynh Mai Secondary School with their answers for the interview and questionnaires

Last but not least, I would like to send my special thanks to my beloved family, who are always supportive and enthusiastic to me during the progress

of making this thesis

ABSTRACT

This thesis is created and completed on the purpose of evaluating the textbook of “Tiếng Anh 6 (10-year Program)”, especially in the context of teaching English at Quynh Mai Secondary School The aim of this thesis is to evaluate the English textbook officially used by the teachers and the students

at Quynh Mai Secondary School Based on the theoretical framework of textbooks evaluation, figures, data and information was collected and analyzed mainly through surveys, questionnaires and interviews The benefits, positive effects, as well as challenges of using the textbooks are analyzed in this thesis Furthermore, some suggestions for overcoming these difficulties are also mentioned

Table of Contents

DECLARATION BY AUTHOR iii

ACKNOWLEDGEMENTS iv

ABSTRACT v

LIST OF TABLES ix

LIST OF FIGURES ix

CHAPTER 1: INTRODUCTION 1

1.1 Rationale 1

1.2 Aims of the study 2

1.3 Research questions 3

1.4 Scope of the study 3

1.5 Significance of the study 4

1.6 Research methods 4

1.7 Structure of the study: 4

CHAPTER 2: LITERATURE REVIEW 6

2.1 ELT Materials in Learning and Teaching Language 6

2.1.1 Position and Purpose of Materials in Teaching Language 6

2.1.2 Types of materials 8

2.2 Models used for Materials Evaluation 10

2.2.1 Cunningsworth’s model (1984) 11

2.2.2 Model of Hutchinson and Water 11

2.2.3 Ellis’s model 14

2.2.4 Summary 15

2.3 Criteria for Materials Evaluation 16

2.3.1 Criteria proposed by Hutchinson and Waters (1993) 17

2.3.2 Criteria suggested by Brown (1995) 18

2.3.3 Criteria suggested by Tomlinson (1998) 18

2.4 Previous Studies on Textbook Evaluation 19

2.4.1 Previous studies carried out in foreign countries 19

2.4.2 Previous studies carried out in Vietnam 22

2.5 Chapter Summary 25

CHAPTER 3: METHODOLOGY 26

3.1 Research Design 26

3.1.1 Rationale 26

3.1.2 Research Instruments 27

3.1.2.1 Document analysis 28

3.1.2.2 Questionnaires 28

3.1.2.3 Interviews 29

3.1.2.4 Classroom Observation 29

3.2 Research Procedure 30

3.3 Data Collection Instrument 31

3.4 Summary 37

CHAPTER 4: FINDINGS AND DISCUSSIONS 38

4.1 Evaluation of Aims 38

4.1.1 Aims stated in the Curriculum 38

4.1.2 Aims stated in the textbook 39

4.2 Evaluation on Methodology 45

4.2.1 Methodology stated in the curriculum 45

4.2.2 Methodology perceived by teachers and students 49

4.2.3 Methodology enacted by teachers 50

4.2.4 Comparison and contrast 57

4.3 Evaluation on Audience 58

4.3.1 Audience stated in the curriculum 58

4.3.2 Audience stated in the textbook 59

4.4 Evaluation on Contents 66

4.4.1 Contents perceived in the curriculum 66

4.4.2 Contents perceived in the Textbook 68

4.4.3 Contents perceived by teachers and students 69

4.5 Findings of the Study 77

4.5.1 Findings related to Research Questions 1 and 2 78

4.6 Discussions 79

4.7 Summary 80

CHAPTER 5: CONCLUSION 83

5.1 Recapitulation 83

5.2 Concluding Remarks 84

5.3 Implications 86

5.4 Limitations and suggestions for the further study 88

5.4.1 Limitations 88

5.4.2 Suggestions 89

REFERENCES 90

APPENDIX 1: LESSON PLAN 93

APPENDIX 2: CITATION OF AIMS OF THE TEXTBOOK TIẾNG ANH 6 98

APPENDIX 3: QUESTIONNAIRE 101

APPENDIX 4: CLASSROOM OBSERVATION 105

APPENDIX 5: INTERVIEW QUESTIONS 107

LIST OF TABLES

Table 1: Judgment on Objectives 41

Table 2: Evaluation on Skills Development 44

Table 3: Evaluation on Methodology 50

Table 4: Evaluation on General Appearance 60

Table 5: Judgment on Design and Illustration 63

Table 6: Judgment on topic appropriateness 65

Table 7: Number of tasks in the book 68

Table 9: Judgment on Content Appropriateness 70

Table 10: Evaluation on Social and Cultural Context 72

Table 11: Evaluation on Methodology 74

Table 12 A Summary of Appropriate and Inappropriate Features of the Targeted Textbook Under Evaluation 80

LIST OF FIGURES Figure 1: The materials evaluation model by Hutchinson and Water 12

Figure 2: Judgment on Objectives 41

Figure 3: Evaluation on Skills Development 43

Figure 4: Evaluation on Methodology 49

Figure 5: Evaluation on General Appearance 60

Figure 6: Details of evaluation on Design and Illustration 62

Figure 7: Judgment on topic appropriateness 64

Figure 8: Judgment on Content Appropriateness 70

Figure 9: Evaluation on Social and Cultural Context 72

Figure 10: Evaluation on Methodology 73

CHAPTER 1: INTRODUCTION

1.1 Rationale

Textbooks are basic materials, compulsory to use in the school, so it is very important to use suitable textbooks in teaching and learning Students are the primary target audience for textbooks From textbooks, students can get access to basic, modern and systematic knowledge Students can acquire knowledge through various sources of information, but knowledge in textbooks is considered as the most standard knowledge In addition to new knowledge, textbooks are also materials that help students consolidate, synthesize, and systematize knowledge through review and study guides

Given the importance mentioned above, the textbook Tiếng Anh 6

(10-year program) was written for innovative purposes to replace the textbook

that has been in used since 2001 English 6 for grade-6-students in Vietnam was developed by the Vietnamese Education Publishing House in accordance with the English Secondary Education Program issued by the Ministry of Education and Training under Decision No 01/QĐ-BGDĐT dated 03 January

2012, followed by English 3, English 4 and English 5 The book is designed

in a way that enables students to use language (phonetics, vocabulary, grammar) to develop communication skills in English through four skills in listening, speaking, reading and writing, with a focus on developing two listening and speaking skills In light of learning-centered approach, pupils are the subject of the teaching process, in which the psychological age of junior high school students, the cultural characteristics of Vietnam as well as of other countries in the world, especially of the English speaking countries, are given special importance

Indeed, the textbook Tiếng Anh 6 for grade-6-students in Vietnam has

been taught experimentally in some classes in secondary schools nationwide for several years However, this material has never been evaluated by teachers who directly teach students in secondary schools to examine if it really meets the defined aims of the compulsory English program for grade-6-students and

if it is suitable to the students’ level of English and their background knowledge Since it was taught experimentally, it has been a subject of much controversy among teaching staff It is said to be well-written but not quite suitable for students who are not equipped with basic knowledge from the elementary level Some teachers argue that each lesson in each unit of the textbook contains a large amount of vocabulary This makes it difficult for students to remember the vocabulary of the lesson In contrast, there are teachers who are enthusiastic about it and say that it is thoughtfully planned

and relevant to current ELT trends in Vietnam For those reasons, “An

evaluation of the textbook Tiếng Anh 6 (10-year program): A case study at Quynh Mai Secondary School”” is chosen as the title of my thesis

Hence, I hope that the findings of the study can help the researcher and other teachers find out the appropriate points as well as the inappropriate points in order to improve or modify the material for future use Especially, I believe that this study can enhance English teaching and learning in schools in Vietnam

1.2 Aims of the study

The study aims at evaluating the textbook Tiếng Anh 6 (10-year

program) to explore if it really meets the defined aims of the course and if it is appropriate to the students’ level of English and their background knowledge

I also hope that the findings of the thesis will partly help to decide whether to

modify any contents or the structure of the book to improve the quality and the effectiveness of the material

1.3 Research questions

The study will be conducted to answer the following research questions:

1 To what extent does the textbook meet the aims of the course?

2 To what extent is the textbook appropriate to the students’ level of English in terms of language and their background knowledge?

1.4 Scope of the study

According to Tom Hutchinson (1993), in textbook evaluation, a number of criteria such as audience, aims, content, methodology, and other criteria should be taken into account However, in this study, due to time constraints and scope of the minor thesis, the main focus of the evaluation is

on the following criteria as audience, aims of the material, content of the material and teaching methodology

The subjects of the study are 80 students chosen randomly from 8 classes: class 6D, 6E, 7D, 7E, 8D, 8E, 9D, 9E (10 students per each class) who studied experimentally the textbook in the school years 2014-2015, 2015-2016, 2016-2017 and 6 teachers of English at Quynh Mai Secondary School, Hai Ba Trung District, Hanoi, Vietnam

The study was implemented from early September 2017 to late May

2018 at Quynh Mai Secondary School as soon as students stabilize their learning at the beginning of the new school year

1.5 Significance of the study

The findings of the thesis are hoped to be useful for the teachers who are teaching English at Secondary Schools The findings of the thesis may provide some suggestions for the authorities and the group of authors of the textbook at the Vietnamese Ministry of Education and Training with scientific evidence to improve the quality of the current material being used for grade-6-students in Vietnam The results also reflect the strengths and weaknesses

of the textbook in use, so that teachers themselves can find appropriate teaching methods for students with different levels, and authorities and authors might also modify the textbook if possible

1.6 Research methods

In order to achieve the purpose of the study and due to the practical

situation that the textbook Tiếng Anh 6 has just been taught experimentally in

schools nationwide for some years, the study was only conducted at Quynh Mai Secondary School in Hai Ba Trung District, Hanoi, Vietnam

In order to answer the two research questions above, the qualitative case study was employed The data were collected through document analysis, questionnaires, interviews and observations with students and teachers at Quynh Mai School

1.7 Structure of the study:

The study is organized into five chapters as follows:

Chapter 1: Introduction

This chapter introduces the background to the study and statement of the problem, the aims of the study, the research questions, the scope of the study, and the significance of the study

Chapter 2: Literature Review

This chapter presents an overview of theoretical issues of textbook evaluation, concentrating on the issues related to ELT textbook evaluation

Chapter 3: Methodology

This chapter describes the data collection instruments and data collection process to conduct the case study

Chapter 4: Findings and Discussions

The results of the data are reported, described, analyzed and discussed Therefore, answers of the two questions are clarified in the chapter

Chapter 5: Conclusion

This chapter provides implications for further studies, and makes conclusion for the thesis

CHAPTER 2: LITERATURE REVIEW

This chapter provides an overview of the background features in the theoretical knowledge for the assessment of a particular Tiếng Anh 6 course’s material Particularly, a general view of the method of using and adapting Tiếng Anh 6 materials allocated to the students at Quynh Mai Secondary School is given and taken as a typical example Overall, there are three sections in this chapter The first section mentions the roles of ELT materials

in teaching and learning the language of English while the second section is a brief review of different approaches to materials evaluation, and the third section consists of some previous researches from predecessors done previously on the evaluation of the text book in different contexts

2.1 ELT Materials in Learning and Teaching Language

2.1.1 Position and Purpose of Materials in Teaching Language

It is widely known that teaching materials are of great importance in teaching and learning any kind of language because it is the most effective communicative teaching method in class; without materials, the teaching procedure in class cannot be conducted According to Cunningsworth (1995), teaching materials are sources of stimulation and ideas which are helpful for activities done in class to provide students sharp practices in communicative interaction Additionally, teaching materials are being functioned as a source

of reference for students on the aspects of vocabulary, grammar and pronunciation as well Furthermore, according to the viewpoint of Dudley Evan and St John (1998), teaching materials can also motivate and encourage students to study by themselves in a more effective way Besides the significant roles to learners, teaching materials are considered a teachers’ support for both inexperienced and experienced teachers; because, generally,

teachers need materials to conduct their lesson Giving an appropriate explanation for this fact, Richards (2001) claimed that teaching materials are considered good ones are able to serve “a form of teacher training” and teachers can use the ideas provided in the materials to plan their own lessons Particularly, according to Dudley Evan and St John (1998), there are four main functions of materials, including language source provision, learning support, motivation source provision and reference source provision First, regarding to language source provision, especially in the situation

of applying a new kind of textbook like Tiếng Anh 6, it can be affirmed that materials are the only method available to get access to the language content that students and teachers have Because of this reason, materials are required

to introduce and explain a real language used in real contexts to be appropriate to a great deal of number of learners and fulfill their needs and request in learning

Second, in terms of learning support, Dudley Evan and St John (1998) once again presented that materials had better involve learners in using their intellectual ideas to apply the language That is the reason why the in-class activities should be “stimulate cognitive not mechanical process” (p.171) Third, with regard to the motivation source provision, it is continued to

be affirmed by Dudley Evan and St John (1998) that materials need to be

“challenging yet achievable to offer new ideas and information whilst being grounded in the learners’ experience and knowledge” (page 172)

Finally, for the aspects that materials are reference source provision; Tomlinson (1998) indicated an opinion that materials are useful when they can help students in class to make “efficient use of the resources in order to

facilitate self-discovery” In other word, the language uses go along with examples, explanation, practice exercises and in-and-out class activities should be provided in the materials It is also necessary to include the detailed explanation and answers for the exercises so that students can use the materials for their self-study at home and teachers can base on these keys and answers to gain some more useful knowledge In the situation of ELT material like Tiếng Anh 6, models of precise and suitable use of language are included and it enables the active teaching process for particular purposes in class

2.1.2 Types of materials

Due to the fact that materials are of great importance to the progress of learning, teachers and scholars, in general, use numerous types of materials; however, there are two most popular types of materials with a large number of smaller types included in, including printed materials such as class textbooks, workbooks, pictures, photos, handout and worksheets and non-printed materials like recorded files and computer-based files (PowerPoint materials for instance) No matter what type of materials they are designed, each kind

of teaching and learning materials has its own function and they are being used differently for numerous learning purposes, which will be discussed further below

2.1.2.1 Textbooks

Textbooks are considered the most important teaching components due

to the fact that they are inevitable learning aids in almost all language teaching programs, and they are often defined basing on the fact that they are materials released and published mainly to be reliable, attractive and user-friendly so that the use of them can bring effectiveness In order to reach this

effectiveness in teaching language, a useful textbook in general, and a course book in particular, has to contain certain factors such as forming a strong course teaching and learning framework which in essence, to build and develop a learning syllabus which is systematically planned and used When allocating textbooks to use, learners study the content presented in it and by following the book map as well as the table of content, students can generalize the main features that they have to study in the book

Nevertheless, according to Harmer (1998), the content of the textbook is not perfect for all of its parts Particularly, despite of the fact that textbooks are well-designed and well-arranged in the content, they still can be difficult and inappropriate to some certain aspects of the action of learning of learners Because the students’ competence may not be suitable to the target language presented in the book Additionally, Swales (2000) also claimed that textbooks which are published with purposes are “less self-sufficient if practice materials and in coverage of skill area” (p.57) Based on this opinion,

it is necessary to supplement the textbook with other appropriate materials to support them and make the learning progress more effectively

2.1.2.2 In-house materials

According to Robinson (1991), in comparison with published materials, in-house materials are more probably to be suitable and specific regarding to the target language and the validity presented in the materials Furthermore, the flexibility of in-house materials, compare to published materials, is also another factor to be considered It is claimed that in-house materials have greater flexibility than published materials owing to their typical characteristics Finally, having improved the drawbacks of the imbalance between textbooks and students’ competence, the author of the in-house

materials is able to make certain that the quality of the teaching method used

in this type of materials is controlled and the knowledge aspects will be appropriate to students Nonetheless, there are some drawbacks of the in-house materials that make them less outweighed than others, and time-consuming can be considered one of those It was observed by Swales (1991) that “the locally produced materials show a striking resemblance to the published materials that have been rejected” (Swales (453, p.6)) In addition

to time-consuming, there is the high expense also be considered, because the progress of in-house materials production requires a large amount of money Furthermore, because of the lack of systematic features, the form of in-house materials is commonly “showers of singe-page handouts” (Swales (453, p.18)); as a result, teaching order can be lost

In spite of the weaknesses they bring, the in-house materials are indicated by Robinson (1991) to have no polar division, in comparison with published materials, which can be very regular to turn into the in-house materials themselves Nonetheless, as mentioned above, creating in-house materials could be high in expense and time-consuming, so it is a tendency in choosing and using existing published materials by teachers to use in class

2.2 Models used for Materials Evaluation

To a great extent, there are some models which can be used for materials evaluation which are proposed by several scholars in the literature However, the popular models which are useful until now are suggested by Cunningsworth (1984), Hutchinson and Waters (1993) and Ellis (1997)

2.2.1 Cunningsworth’s model (1984)

According to Cunningsworth (1984), there are aspects for materials evaluation, including linguistic elements of the materials regarding topics, content of language, methodology, skills and so forth

It is noted by Cunningsworth (1984) that language content used for the assessment is better if it is concentrated on the suitability of the vocabulary, grammars, functions and structures illustrated in the materials Another viewpoint is also presented by the author is that it is necessary to be interesting in terms of the materials’ topics, and sophisticated and various content are significant as well Additionally, the students should be taken advantages by expanding the knowledge of the language level of these topics

In terms of the skills which are introduced in the materials, the issues about whether the skills of listening, speaking, reading and writing are covered enough as well as the question about whether the skills are appropriate to the target of the course and the syllabus requirements should be solved Furthermore, the methodology of the materials is also being concerned; in particular, the suitability of the materials to the situation of teaching and learning and the name

of the techniques used to introduce novel language items

2.2.2 Model of Hutchinson and Water

It is noted by Hutchinson and Water (1993) that evaluation process of materials is a matter in which the fitness of a particular issue for a specific purpose is judged In essence, materials evaluation is a matching process: matching should come with solutions From this perspective, an evaluation could be divided into four main steps, including defining criteria, subjective analysis, and objective analysis and matching

The relationship among these steps is illustrated in this graph:

Figure 1: The materials evaluation model by Hutchinson and Water

It can be seen from the model that a logical procedure of the evaluation progress for the materials is provided, which has a purpose of examining the appropriateness of the materials for a particular students group In the first place, the criteria for the evaluation were set out by the evaluator, who could also mention his or her favorite criteria In the second step of the process, the subjective analysis was determined, and it is time for the course’s requirements relating to the criteria to be analyzed It is also noted by Hutchinson and Water (1993) that the subjective analysis should not be regarded as an irreplaceable requirement, and the evaluator should use the process of materials evaluation as a method of examining and developing his

or her ideas single-handedly to the requirement Subsequently, the evaluator

Defining Criteria The bases in which the materials will be

judged The more significant criteria

Subjective analysis

The realizations of the

criteria that are wanted

in the course

Objective analysis The way the materials are being evaluated and how they realize the criteria

Matching The level of consistency between the

need and the materials

needs to conduct the next step – Objective analysis, which has the function of analyzing the materials basing on selected criteria Eventually, the findings of all the previous steps were taken on the purpose of comparing to materials requirements at the step four with a view to deciding and evaluating the consistency between the requirements and the materials As can be seen from this model, three options may take place after the process of evaluation, the materials may not fulfill any requirement at all, or just match the requirements partly In these situations, especially when some criteria of the requirements are partially mismatch with the materials, according to Hutchinson and Waters, the evaluator can think about the possibility of considering the more prominent criteria to the learning object concerned such as teachers, students

or sponsors and take the criteria which are easier to adapt

It is also noted by Hutchinson and Waters (1993) that a checklist of criteria was proposed single-handedly for objective and subjective analysis, which had a function of assessing English teaching materials by concentrating

on typical issues namely the aims, the methodology, the audience and the content:

 The aims: the evaluator is in charge of checking whether the materials’ aims match with the whole objective of the course or not

 The methodology: This criterion is about some important aspects such as the learning theories, the attitude of the students to learning a foreign language like English, types of possible exercises or tasks, techniques of teaching and learning effectively, learning facilities, suitable guidance or support for a better teaching and the materials’ flexibility

 The audience: Different information are analyzed by the evaluator, including the age, gender interests, English competence and educational background of the students

 The Content: A certain amount of sub-criteria should be examined; including language aspects and description, micro and macro-skills and their percentage, text types, organization of the content, topics and sequence in order to assess whether the materials are appropriate

to the needs of the students or not

2.2.3 Ellis’s model

Ellis (1997) claims that language evaluation in teaching and learning, in nature, has been primarily predictive and has concentrated on all of the materials concerned in the set of the materials which are retrospectively applied on a regular basis As a result, a model was suggested which could be conducted as a sequence of micro-evaluations In other words, a micro-evaluation is the evaluation of a teaching mission in a very particular information which was interested in by the evaluator Particularly, in this model, it was suggested by Ellis that a detailed empirical assessment was practiced and also an equivalent task level of evaluation which has a relation

to its actual context of teaching and learning was concentrated Broadly speaking, this model has an aim to recognize the consistency between planned and used tasks, and it was also suggested by Ellis (1997) that some factors concentrated macro-evaluation are able to be applicable to micro-evaluation; including approach, types of information, timing, focus, purposes and scope evaluators; and to clarify it, Ellis (1997) suggested seven steps for the evaluation, including:

 Selecting an appropriate task to follow;

 Illustrating the task which requests the content’s specification based

on these factors: input, language activities, procedures and outcomes;

performing the task and the results of this performance and the opinions of teachers and students about the tasks

 Qualitative and quantitative analysis of the collected data, figure and information

 Giving a conclusion basing on the discovery from the task evaluation and recommendations for future learning and teaching process

 Writing a report

From Ellis (1997)’s perspective, a micro-evaluation could be considered

to be more manageable than, when compared to, a macro-evaluation Nevertheless, it may have some weaknesses such as time-consuming as well

as labor-consuming, when it requires much effort to provide a basis for macro-evaluation subsequent process; furthermore, this kind of evaluation are only carried out in the scope of the classroom, when the target materials are applied

2.2.4 Summary

From the analysis of the three researched model, it can be concluded that the three above-presented approaches suggest three alternatives to conduct materials evaluation Particularly, Hutchinson and Water’s model (1993) consists of the subjective analysis, which identifies the requirements of the learning materials of the course, and objective analysis, which meanwhile investigates the realization of the requirement of the learning course

Besides, Ellis (1997) suggested the model of micro-evaluation and macro-evaluation, which has their own advantages and drawbacks but are interdependent because the existence of a sequence of micro-evaluations is the provision of the bases for subsequent macro-evaluation, and this process often are time-consuming and labor-consuming

Finally, Cunningsworth (1984) offered a model which is relatively detailed with features of macro complemented, and its aims is support the teacher to make the most appropriate decisions when choosing materials to use in class Among the models which are discussed above, the one which was suggested by Hutchinson and Waters (1993) shows the materials evaluation procedure in the most logical and clearest way of presentation Owing to this reason, Hutchinson and Water’s model was selected to evaluate the materials: English 6 at Quynh Mai Secondary School

2.3 Criteria for Materials Evaluation

In order to conduct any evaluation, it is crucial for evaluator to take definition of criteria for evaluation into account, because they have a function

of being the bases for the evaluators to depend on when making assessment According to Tomlinson (1998), criteria are things that evaluators use in order

to “reach a decision regarding what needs to be evaluated” Additionally, it was suggested by Ellis (1997) that checklists are traditionally used to evaluate materials in a predicative way and determine the appropriateness of the materials in certain contexts of teaching It was also noted by Dudley-Evans and John (1998) that criteria for the evaluation regularly depend on what and why this kind of materials should be assessed Because of the variety of materials, there are also numerous of criteria sets proposed by different

scholars, including Hutchinson and Waters (1993), Cunningsworth (1995), Brown (1995) and Tomlinson (1998)

2.3.1 Criteria proposed by Hutchinson and Waters (1993)

Materials evaluation is defined from Hutchinson and Waters’ perspective

as a “matter of judging the fitness of something for a particular purpose” In order to examine the appropriateness to the particular group of learners of the materials, it is vital to set up a group of criteria, whose purpose is evaluating the materials with four main focuses, against which the assessment is based

 Audience

According to Hutchinson and Waters (1993), Tiếng Anh 6 (10-year Program) is a learning-centered material which has an overall aim of meeting the needs of a group of students; in particular, the secondary students at Quynh Mai Secondary School The textbook Tiếng Anh 6 (10-year Program) has to, therefore, be designed according to the students’ specific needs Therefore, it is necessary for the evaluator to find out throughout about the information of students to examine whether the textbook Tiếng Anh 6 (10-year Program) is suitable to the age, knowledge of English, educational background and professional competence or not

 Aims

In every course of language, an evaluation of textbooks needs to be conducted so as to the consistency between the objective of the course and the textbooks is found out

 Content

Content of the textbooks is considered a vital factor; accordingly, it is essential to examine whether important elements in the textbooks such as

language items, micro-skills and macro-skills, text types or topics match with the needs of the students

Textbooks are designed with their methodological background, which is the reason why it is necessary for the evaluator to find out whether the theoretical background in the textbooks is appropriate to the equivalent aspects of the education program, including competence of the students, satisfaction of teachers and students about the teaching techniques conveyed

in the textbooks

2.3.2 Criteria suggested by Brown (1995)

Another checklist for the evaluation of textbooks was also suggest by Brown (1995) From his perspective, the materials; textbooks in particular; is considered from five aspects, including teaching ability, curriculum fit, background, physical and logical characteristics Nevertheless, most of them concentrate on external evaluation The other aspects are not illustrated in detail, for example, content and methodology of the materials

2.3.3 Criteria suggested by Tomlinson (1998)

The set of criteria proposed by Tomlinson (1998) focused on making general assessment on the textbooks in an impressionistic way and examining the content of the textbooks in-depth Accordingly, a close analysis of materials, textbooks in particular was offered, which proven to be a preliminary step to the evaluation off the textbooks and research in class From the scholar’s perspective, researching the textbooks in a very careful way means that these questions given below should be answered:

 Question 3: How can the findings be related to the particular context

of teaching?

Numerous aspects are mentioned by Tomlinson (1998) and teachers or analysts can be based on these aspects, which are divided into two main sections: design and publication, to examine sets or materials In particular, publication is related to the physical aspects of the textbooks and how these aspects appear as a set of books for students to learn, and the design reflects a very clear viewpoint of the scholar that materials, textbooks in particular, is focused as a pedagogical advice; in other words, it is an aid to the process of learning and teaching a foreign language Additionally, the author has an opposite view about the three levels of analysis, which are checked through the physical elements of the textbooks, through deduction about the needs which are likely to be created for the teachers and students to generalize the apparent principles of the textbooks which are underlying From the scholar’s perspective, these three aspects of analysis would be able for teacher-analysts

to check a wide range of elements of the textbooks, and they are mode of practice, idea and language

2.4 Previous Studies on Textbook Evaluation

Owing to its importance in teaching and learning language, textbooks evaluation has been drawing much attention from researchers As a result, there are several theses conducted in order to evaluate currently used textbooks, and each dissertation or research apply different ways of approach

to evaluate the textbooks, but they have a similarity that they all describe the way of evaluating the textbooks and how this process provides feedbacks for the modification and enhancement of the textbooks

2.4.1 Previous studies carried out in foreign countries

Tomlinson (2001) evaluated some textbooks published by leading UK

publishers: "Language in Use" by Cambridge University Press The overall

criteria and course book-specific criteria were carefully treated under four categories Moreover, specific criteria for cassettes and CD ROM, teacher's book, workbook and video were also used in this evaluative study The author commented on every single course book and drew seven positive trends and sixteen negative trends in the UK's current courses

Another scholar, Alamri (1998), conducted a study to evaluate the Sixth Grade English Language Textbook for Saudi Boys' Schools A 63-item questionnaire was grouped under 12 main categories: the general appearance, design and illustration, accompanying materials, objectives, topic appropriateness, language components, socio-cultural contexts, skills development, teachability, flexibility, teaching methods, and practice and testing The findings of this study revealed that this is a good textbook because it satisfies almost all the categories listed Ten points of recommendations emphasizing the characteristics of a good textbook is also made in terms of glossary, attractive illustrations, critical thinking topics, better listening materials, students' multi-intelligences consideration and so

on

In general, it can be seen from the two above-mentioned evaluation from the two foreign scholars is that the significant data resources which can be used for the evaluation process can be collected from various methods Particularly, it can be seen from the evaluation of the first author is that there were many factors that have a very close connection to the textbooks and they can create the great effect about the effectiveness of the textbooks Meanwhile, it is also noted from the evaluation of Alamri that teachers are the ones who can be very helpful in giving practical recommendations for the

improvement of the textbooks The evaluation of Alamri also used a method

of collecting data by using documents and records, involving documents planning, needs statements, objectives and targets of the education program, lesson records of teachers and so forth

2.4.2 Previous studies carried out in Vietnam

In 2006, the researcher Nguyễn Thanh Liêm conducted an evaluation of the textbooks, course book in particular, of Television specialist for the students at the College of Television The researcher depended on the theoretical background knowledge suggested by Hutchinson and Waters (1993), Cunningsworth (1995) and Celce – Murcia (2001) to carry out her evaluation

In 2009, the researcher Lê Thị Huệ carried out a case study about the evaluation of the textbook Tiếng Anh 11 taught at Phan Dinh Phung High School In her minor program thesis, she analyzed the types of textbooks evaluation, including pre-use evaluation, in-use evaluation and post-evaluation; textbooks evaluators and methods of textbooks evaluation

Undoubtedly, there is a common way of researching among these authors that it is essential to collect data from students It can be seen from the study of Nguyễn Thanh Liêm that students, who are able to give “evidence”

of what they have gained from the lessons and the textbooks as well, are considered as the main source of data and precious information, especially in

a summative evaluation Similarly, the scholar Le Thi Hue use the collective method for her study because of the fact that by getting information from the students, besides the data relating to the needs of students, the evaluators can also collect the attitude towards English itself and the textbooks, learning motivation, preference and viewpoint for the textbooks themselves

In 2010, another researcher, Nguyễn Thị Hồng Châm, proposed another minor program thesis about adapting English 10 textbook towards promoting learner’s autonomy and communicative competence in learning grammar, a very helpful and useful for teacher in using the textbook of English 10

Nguyễn Thị Hương Lan (2004) made an evaluation of the piloted

"English 10 textbook 2" at Nguyen Tat Thanh high school The author used

the model suggested by Hutchinson and Waters (1993) which was reflected in

a twenty-item questionnaire for teachers and another twenty-two item questionnaire for students and informal interviews with the teacher informants and the student sample to evaluate the textbook's objectives, content and methodology as compared to the statement objectives and aims defined by the Vietnam's Ministry of Education and Training (MOET) for grade 10 The findings of this study show that the textbook under evaluation partly met the course requirements and suitable to the students' level of English in such areas

as suitable topics, appropriate listening sections, various and appropriate text types and skill-integrated task types and activities However, students' ability

to communicate in English about the required topics stated in the objectives are not well-met; the texts on speaking, reading and writing skills are difficult and there is not enough proper instructions given Some adjustment of content and methodology, exercises designed basis and adaptation techniques such as adding, deleting, simplifying and reordering are also included in the study

Trần Thị Chung Oanh (2010) evaluated the material "Lifelines" for the

first year non-English major students at Hai Phong University The author adopted Hutchison and Waters (1987)'s criteria and employed questionnaires and informal interviews as research instruments Major findings of the study revealed that the material's content and methodology suited the students' requirements but there was some mis-match in time allocation Finally, suggestions on the material improvements on time proportion for each unit and time allocated for the course were made

All of the authors mentioned above used the similar way of evaluation,

in which teachers are the people who play an important role in evaluation

process, because on the purpose of getting valuable data, it is believed that similar piece of information should be collected from variety of sources and it

is necessary for both teachers and learners to involve in the whole process of evaluation due to the fact that they are the object who work with the textbooks

in the whole education program; therefore, they will be able to name and cope with the textbooks during the time they work with the textbooks and subsequently give practical recommendations for making improvement Nevertheless, it is also noted from these studies that teachers often have a feeling that any resultant evaluation documents will be negatively critical

Nguyễn Thị Nguyệt Ánh (2010) conducted a study on teachers

evaluation on "Let's go 1A - 2nd edition" for grade 3 at primary schools in the

north of Viet Nam A combination of the evaluation model by Hutchison and Waters (1987) and Cunningsworth (1995)'s checklist was employed The criteria were divided into six areas: objectives, content, language points, accompanying supplementary materials, general look and methodology The results of the survey indicated that "Let's go 1A second edition" basically met the objectives of the course and the curriculum Limitations of the book rested

in limited topics, unbalanced skills, a challenging communicative methodology for teachers and the way linguistic input was presented Solutions to these problems were the presentation of communicative goals in course syllabus design as well as adaptation techniques

For short, while some of the checklists focus on such aspects of evaluation as physical appearance or pedagogical methods, others take into account the socio-cultural facets All of the authors mentioned used one single checklist or a combination of two or more checklists of criteria, depending on their specific purposes In the current study, it is decided to select features in these checklists which are common and at the same time necessary to evaluate

all the necessary considerations of the book in terms of not only its physical appearance, but also the pedagogical methods presented, social-cultural context and some other important values

2.5 Chapter Summary

From the above-mentioned analysis and evidence, the literature presented in this Chapter could help to choose the most appropriate research methodology to conduct the thesis In shorter words, the literature review in Chapter 2 has just illustrated the theoretical background knowledge to textbooks evaluation The main things mentioned in this chapter consists of reasons for textbooks evaluation, definition of textbooks evaluation, types of textbooks evaluation, three models for textbooks evaluation, and criteria for the process of evaluation, data collecting instruments for textbooks evaluation and previous textbooks evaluation researches It is hoped that the reviewed literature mentioned above could be able to provide a methodological and theoretical framework for this evaluation research

CHAPTER 3: METHODOLOGY

In this chapter, a description of the methodology in this thesis was introduced As mentioned in the introduction, the study goals are to seek the answers to the following research questions:

1 To what extent does the textbook meet the aims of the course?

2 To what extent is the textbook appropriate to the students’ level of English in terms of language and their background knowledge?

There are three main sections in this chapter The first section presents the rationale for the research design, and provides a description of the context

as well as the tools for gathering data for the purpose of the study, including the investigation of the document, questionnaire and interview The second section introduces a short description of the participants of the study and the research procedure, the last section introduces process of collecting the data

It was noted by many scholars such as Teddlier (1973) that this multi-method approach is to make certain of a clearer view into discrepant level of analysis

3.1 Research Design

3.1.1 Rationale

This study is carried out at Quynh Mai Secondary School, Hai Ba Trung District, Hanoi City In this school, English has been taught as a compulsory subject for all students The book used for the 6th-grade students is “Tiếng Anh 6”, published by Vietnam Education Publishing House

Students at Quynh Mai Secondary School, Hai Ba Trung District, Hanoi City have various competence of English, some of them are excellent but most of the student are normal in using English Therefore, this research

was aimed to find out the way of teaching and learning English by evaluating the textbook and suggest some effective suggestions

The method used in this study is qualitative research, which is a type of scientific research In general terms, this method searches for answers to a question, methodically uses a predefined set of actions to answer the question, gathers evidence and produces findings that were not determined in advance Furthermore, it searches for to understand a given research problem or topic from the viewpoints of the local population it involves Qualitative research is especially applicable in getting culturally specific information about the values, opinions, manners, and social contexts of particular populations

In this study, the researcher used the case study design, which is defined

as an empirical inquiry that examines a current phenomenon within its life context, especially when the borders between phenomenon and context are not clearly obvious Case study is a way of investigating an observed topic

real-by following a set of pre-specified procedures, and that was the reason why the researcher decided to use it Furthermore, the researcher was dealing with

a single case, looking at a specific grade and a specific number of participants

in a particular area and community, and case study was appropriate to fulfill the requirement

3.1.2 Research Instruments

In general, the field evaluation and textbooks assessment in particular have been dominated by scientific methods, which, according to Taylor and Steel (1996), seek to set up causes and effects relationships, give a generalizable final conclusion and provide quantitative and qualitative data, for a long time Nonetheless, it can be noted from the viewpoints of several scholars including Robinson (1991), Hutchinson and Waters (1993) and

Brown (1995) that some necessary instruments, such as document analysis, questionnaires, interview, assessment or checklists are being used once or several times in most of the evaluation of textbooks, which will be discussed below

3.1.2.1 Document analysis

According to Taylor (1991), document analysis is the using “content analysis and other techniques to analyze and summarize printed materials and existing information” Generally, the aim of the use of the instrument is to assess the current data that are possible to find out, and, according to Brown (1995), the document need analyzing can include “extra data sources” or

“data sources within a program”; including materials in the progress of the evaluation, syllabus of the education program, the curriculum It is also noted

by Nunan (1992) and Brown (1995), this kind of the instrument is used to get the qualitative information that “concerned with qualities and non-numerical characteristics.”

3.1.2.2 Questionnaires

According to Nunan (1992), questionnaire is a common method of data collection, which enables the researcher to get necessary information in field settings and this kind of data are more amenable to quantification than kind of discursive data such as oral language transcription or journals of participants’ observer Additionally, according to Lynch (1996), questionnaire is considered the most effective kind of written interview, so they are often used

to substitute for the interview process if there is not much time and resources Obviously, the greatest benefits of this kind of instrument could be mentioned are that the data got from questionnaires are less time-consuming than interviews, and they can be collected on a large scale as well Nevertheless,

questionnaires themselves still contain some drawbacks that need considering such as the unfinished questions or the loss of the questionnaire papers A questionnaire is often constructed with many closed-ended questions which require only Yes/No response of item selection and opened-ended questions which require participants to give their own opinions or comments, which easily cause boredom when answering

3.1.2.3 Interviews

It was mentioned by Taylor (1996) that interview is a method of collecting data by having conversation with people chosen randomly or intentionally and listening to their opinion about the issue related Interviews could be conducted under several ways, from the formal interviews which highly structured to normal ones with free-flowing way of speaking In terms

of the types of an interview, it is noted that there are several forms of interviews, including structured, semi-structured and unstructured interview Among these types, the semi-instructed interview is considered the most popular instrument regarding to interpretive research According to Brown (1995), semi-structured interview is able to collect more data than the others

To a great extent, interviews, based on the viewpoint from Hussey (1997); Robinson (1991) and Brown (1995), is a method of collecting data which induce everything become easier to make comparison among given answers because it can be conducted by face-to-face, screen-to-screen or voice-to-voice conversation; as a result, despite being claimed to be time-consuming, these types can help the interviewer to collect participants’ viewpoint in a real way

3.1.2.4 Classroom Observation

It was defined by Robinson (1991) that “a classroom observation is much shorter than a questionnaire” In particular, an observation contains a certain number of questions which could be closed or open ended Furthermore, it was also noted by Robinson (1991) that this instrument type could be used then the concentration of the evaluation is “small-scale aspects

of a program, for example one component, such as speaking skills, of a syllabus.”

On the other hand, an observation is considered to be another type of assessment According to Taylor (1996), observation is the result of a set of standards which are established so as to evaluate knowledge, performance, skill or skills test and the assessment could be conducted by evaluators or teachers It is also noted by Brown (1995) that by using observation, much information about “the general ability levels of the students” and “possible ability groupings that will make sense within a program” can be collected, although designing a test in a high quality is not a work that can be done easily

3.2 Research Procedure

The researcher of this thesis and the participants of this study were students and teachers from Quynh Mai Secondary School in the academic year of 2017 – 2018 The total number of the study population is 86, including

80 secondary students from grade 6 to grade 9 and six lower secondary teachers While all the teachers teaching the textbook are chosen, ten students from each of the six classes were selected randomly to participate in the procedure Questionnaires were sent to all the teachers and students and all of them were collected and used in this study

Another feature about the participant is all of the teachers involved in the survey are female, ageing from 28 to 50 They have been teaching English for at least seven years and all of them have got B2 or C1 certificates (according to CFFR) to be qualified for teaching these classes

In terms of the 80 student participants, there are 35 female students and 45 male students varying from 12 to 15 years old The students' English competence is at the qualified level at A1 and A2, some of them are B1

3.3 Data Collection Instrument

As presented in the literature review, important data for the process of evaluation can be delivered by several instruments which are used to collect data for the procedure of textbook evaluation In the scope of the research, the chosen instruments are document analysis, questionnaires, interviews and observations

Particularly, on the purpose of collecting data, the investigator utilized

two questionnaires with yes-no questions in for teachers and students to evaluate Furthermore, talks with the teachers and students were also conducted after the collection of the information from the survey with a view

to getting more in-depth data

Additionally, the questionnaires made based on the checklist of standards which are the most appropriate to the research Subsequently, they were delivered to the teacher and students at Quynh Mai Secondary schools at

the same time when they have completed the lessons in the textbook "Tiếng

Anh 6” The research data was gathered via the questionnaires delivered to

the teachers and students