Đánh giá sự phù hợp về nội dung của bài kiểm tra tiếng Anh cuối kỳ dành cho sinh viên không chuyên năm thứ hai của Trường Đại học Dân lập Phương Đông

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF FOREIGN LANGUAGES AND INTERNATIONAL STUDIES FALCUTY OF POST-GRADUATE STUDIES ……………… ***………………… TRẦN THÚY QUỲNH THE CONTENT VALIDITY OF THE CURRENT ENGLISH ACHIEVEMENT TEST FOR SECOND YEAR NON MAJOR STUDENTS AT PHUONG DONG UNIVERSITY (Đánh giá phù hợp nội dung kiểm tra tiếng Anh cuối kỳ dành cho sinh viên không chuyên năm thứ hai Trường Đại học dân lập Phương Đông) M.A MINOR THESIS Field: ENGLISH TEACHING METHODOLOGY Code: 60-14-10 Course: 18 (2009-2011) Supervisor: M.A Kim Van Tat HA NOI- SEPTEMBER 2011 iv TABLE OF CONTENTS Page Acknowledgement……………………………………………………… i Abstract………………………………………………………………… ii List of tables and figures…………………………………………………iii The table of contents………………………………………………… iv Chapter 1: Introduction 1.1 Rationale………………………………………………………… 1.2 Scope of study…………………………………………………….2 1.3 Aims of study…………………………………………………… 1.4 Methods of study………………………………………………….2 1.5 Research questions……………………………………………… 1.6 Design of study……………………………………………………3 Chapter 2: Literature review 2.1 Language testing 2.1.1 Definition of language testing…………………………………… 2.1.2 The roles of language testing…………………………………… 2.1.3 Relationship between testing and teaching- learning…………… 2.2 Major Characteristics of a good test 2.2.1 Test validity……………………………………………………….8 2.2.1.1 What is test validity? 2.2.1.2 Types of test validity………………………………………… 2.2.1.2 Face validity…………………………………………….9 2.2.1.2 Content validity……………………………………… 10 2.2.1.2.1 What is content validity? 10 2.2.1.2 How to make the test more valid? 11 2.2.2 Test reliability…………………………………………………… 13 v 2.2.3 Relationship between reliability and validity…………………… 16 2.2.4 Practicality…………………………………………………………17 2.2.5 Discrimination………………………………………………17 Chapter 3: The study 3.1 English learning, teaching and testing at Phuong Dong University 3.1.1 The students……………………………………………………… 19 3.1.2 The teachers……………………….……………………………….19 3.1.3 The course book “New Headway Elementary- The third edition” 19 3.1.4 Syllabus and its objectives……………………………………… 20 3.1.5 The final achievement test…………………………………………20 3.2 Research method……………………………………………………20 3.2.1 The survey questionnaires…………………………………………21 3.2.2 Document analysis……………………………………………… 21 3.3 Data analysis……………………………………………………… 22 3.3.1 Analysis of the final achievement test…………………………… 22 3.3.2 Analysis of the survey questionnaire for students…………………26 3.3.3 Analysis of the survey questionnaire for teachers…………………30 3.4 Results……………………………………………………………….32 Chapter 4: Recommendations and conclusions 4.1 Recommendations………………………………………………… 34 4.2 Conclusion …………………… ………………………………… 43 4.3 Limitations………………………….……………………………… 43 References…………………………………………………… 45 Appendixes Appendix 1: The content of the course book……………… I Appendix 2: Survey questionnaires for students………… IV Appendix 3: Survey questionnaires for teachers……………V Appendix 4: Answer key for reading task ………………….VII Appendix 5: Answer key for the new final achievement test.VIII iii LIST OF TABLES AND CHARTS Table 1: Scores on test A (invented data) by Arthur Hughes Table 1: Scores on test B (invented data) by Arthur Hughes Table 3: The components of the final achievement test Table 4: What students had been taught and what they had been checked in part I, II, III of the test Table 5: What students had been taught and checked in the writing part Table 6: Paper specification grids for the final achievement test Chart 1: Students’ comment on validity of the test Chart 2: Students' comment on time allowance of the test Chart 3: Students' comment on difficult level of the test 10 Chart 4: The result of the test 11 Chart 5: The purpose of the test Chapter 1: Introduction 1 Rationale These days, the need of learning English has become greater and greater In our country Viet Nam, having recognized the importance of it, the Ministry of Education and Training (MOET) has recently decided that English is a compulsory subject in most high schools and universities This decision requires both teachers and students to alter their ways of teaching and learning In addition, testing is one effective way to evaluate teaching and learning They are so closely related Testing validates the teaching-learning process while teaching and learning provides a great source of language materials for testing to exploit And testing is a concerned matter to all teachers During the teaching time at Phuong Dong University, the writer heard both teachers and students here complaining that the English test did not often faithfully reflect the teaching and learning process or in other words, the test did not reflect what the students learnt and what the teachers taught What was tested was not really taught and the test measures neither the achievement of the course objectives nor the expected skills and knowledge of students It is shared by some test researchers as Brown (1994: 373) and Hughes (1989:1) on recent language testing: “A great deal of language testing is of very poor quality Too often language testing has a harmful effect on teaching and learning and too often they fail to measure accurately whatever it is they are intended to measure.” Another reason for the selection of this research topic lays in the fact that language testing at Phuong Dong University has not been paid enough attention to Classroom language tests were often written in a hurry because the teachers here could not find time to think carefully and plan the test Sometimes, they did not have a clear idea of what they were testing students for and why They were busy mixing the number of various question types and from that many students got low marks Due to its close relationship with language teaching and learning, testing deserves proper attention from teachers and students in order for a positive backwash on the teachers‟ teaching, students‟ satisfaction and encouragement in their study In order to design a good test to have exact, fair and effective evaluation of students‟ knowledge and performance of English, teachers are supposed to have good knowledge of test writing techniques and testing theories Because of all above-mentioned reasons, the writer is encouraged to undertake this study entitled: “Content validity of the current English achievement test for second-yearnon-major students of English at Phuong Dong University” with aims at finding out the strengths and weaknesses of this test in terms of the content validity and some, if any, suggested solutions for the improvement of it 1.2 Scope of study The scope of this thesis is limited to a research on evaluating the final achievement test in terms of its content validity by comparing the objectives, the syllabus and the textbook allocation with the test contents The study provides investigated and analyzed data of the currently used test and proposes practical suggestions on the improvements of this test Due to the limitations of time, ability and conditions, it is impossible for the writer to cover all the tests Only some suggestions for the improvements of the test are presented 1.3 Aims of study The study aims at checking the content validity of the final achievement test for second-year-non-major students at Phuong Dong University It places high emphasis on analyzing the contents of the final achievement test The specific aims of this research are: - To find out the strengths and weaknesses of the currently used test with reference to the content validity - To suggest some improvements for the test 1.4 Methods of study In order to achieve the above-mentioned aims, a combination of many methodologies was utilized Firstly, the writer based herself both on the theories and principles of language testing and major characteristics of a good test with a special focus on test content validity From her own reading, many reference materials have been gathered and analyzed to draw out a theoretical basis to evaluate the achievement test being used for second year students in terms of its content validity Basing on what students had learnt in their first semester and the contents of this test, the writer would examine its content validity In addition, qualitative methodologies involving data collected through survey questionnaires were employed Two sets of questionnaires were administered to both English teachers and students at Phuong Dong University to investigate their evaluative comments on the content validity of the final achievement test and some suggestions for its improvements 1.5 Research questions In this study, the writer tries to answer the two following questions: Question 1: What are the strengths and weaknesses of the final achievement test with reference to the content validity for second year non major students at Phuong Dong University? Question 2: What are some suggested solutions for the improvements of the test? 1.6 Design of study The thesis is organized into five major chapters: Chapter INTRODUCTION presents such basic information as: the rationale, the aims, the methods, the research questions and the design of the study Chapter LITERATURE REVIEW presents a review of related literature that provides the theoretical basis for evaluating and building a good language test This review includes background on language testing, criteria of good tests and theoretical issues on test content validity Chapter THE STUDY mentions the methods used in the research and which shows its detailed results of the surveys including the questionnaires and the analysis of the final achievement test in order to find out its problems with reference to the content validity Chapter RECOMMENDATIONS AND CONCLUSIONS Recommendations provide some suggestions for the improvements of the final achievement test basing on the mentioned theoretical and practical study Conclusions summarize the matters of research, its findings as well as its limitations Chapter 2: Literature review This chapter provides an overview of the theoretical background of the study It includes three main sections 2.1 Language testing 2.1.1 Definition of language testing Testing is an important part of every teaching and learning experience and becomes one of the main aspects of methodology The issue of language testing and its significant role has been discussed a great deal by many professionals and research worldwide Different definitions of language testing are given out with various points of view According to Allen (1974:313), testing as an instrument to ensure that students have a sense of competition rather than to know how good their performance is and in which condition a test can take place He says: “Test is a measuring device which we use when we want to compare an individual with other individuals who belongs to the same group.” Carroll (1986:46) stresses a psychological or educational test is a procedure designed to elicit certain behavior from which one can make inferences about certain characteristics of an individual In other words, a test is a measurement instrument designed to elicit a particular behavior of each individual According to Bachman (1990:20), what distinguishes a test from other types of measurement is that it is designed to obtain specific sample of behavior This distinction is believed to be of great importance as it reflects the primary justification for the use of language and has implications for how we design, develop, and use them to their best use Thus, language tests can provide the means for more focus on the specific assure of interest In the point of view of Ibe (1981:1), “a sample of behavior under the control of specified conditions aims toward providing a basis for performing judgment.” The term a sample of behavior used here is quite board and it means something else rather than the traditional types of paper and pencils Yet, Heaton (1988:5) has different opinion In his ideal, tests are considered as a mean of assessing the students‟ performance and to motivate the students He looks at tests with positive eyes as many students are eager to take tests at the end of the semester to know how much knowledge they have One important thing is that he points out the relationship between testing and teaching 2.1.2 The roles of language testing Language testing is a form of measurement It helps the teachers: + To assess the learner‟s achievement in a language program, for example, to evaluate the testee‟s language knowledge in relation to a given curriculum or material which the testee has gone through in a given course + To assess a learner‟s proficiency in language in relation to future language use; for example, to find out if a person‟s language is good enough for him to become a tourist guide This is the future use of the language regardless of what language programs or materials the testee went through + To diagnose a learner‟s strengths and weaknesses in a language and to attempt to explain why the certain problems occur and what treatments could be used to tackle these problems + To classify or place the testees in the appropriate language classes + To measure the testee‟s aptitude for learning a language + To evaluate the effectiveness of a language program This is often done by using experimental and control classes with the same educational objectives but using different methods and materials to achieve these objectives, Brown (2000:5) In another way, Bebecca.M.Valette (1977:3) comments that classroom tests play three important roles in second language teaching program They are defining course objectives, stimulating student progress and evaluating class achievement Firstly, classroom tests help us to define the course objectives Students are quick to observe types of tests given and to study accordingly Thus, much as the teacher may emphasize oral fluency in the classroom, if any tests are written tests the students will soon concentrate on perfecting the skills of reading and writing Secondly, tests help stimulating student progress As much as possible, the time given over to classroom testing should provide a rewarding experience The test should furnish an opportunity for the students to show how well they can handle the specific elements of the target language; gone are the days when the teacher designed a test to point up the students‟ ignorance or lack of application Tests should be distinctly announced in advance to permit the students to prepare adequately If the students themselves are expected to demonstrate their abilities, it is only proper that they should learn as soon as possible after the test how well they did The test best fulfills its functions as a part of the learning process if the correct performance is immediately confirmed and the errors are pointed out The last role of testing is evaluating class achievement Through frequent testing, the teacher can determine which aspects of the program are presenting difficulties for individual students and for the class as a whole By analyzing the mistakes made on a given test, the teacher can determine where to concentrate extra class drills and how best to assist each student At the same time, testing enables the teacher to discover whether the class objectives are being met Through tests, the teacher can evaluate the effectiveness of a new teaching method, of a different approach to a difficult pattern, or of new materials The most familiar role of the classroom test is to furnish an objective evaluation of each student‟s progress: his or her attainment of course objectives and his or her performances in relation to the rest of the class 2.1.3 Relationship between testing and teaching- learning In the past, teaching and testing used to separate both theoretically and practically According to Williams (1983), a test is necessary imposition but outside the classroom, it is unpleasant one because of two main reasons The first one is that testing is concerned with competition rather than cooperation Thus, while classroom activities may involve pair works and group works, such cooperation during a test is condemned as copying, and the individual is expected to work alone If these are perfectly possible, the results of a group test may tell us very little about each individual in that group In the same way, testing does not admit cooperation between teachers and learners The teacher who helps and encourages the learners with their tasks and responds to their difficulties, in a test situation, withdraws cooperation The other reason followed from the first is that there should be a winner and loser in the test To be sure, those who close to win themselves not feel too upset, but those who gain little from experience, may feel conscious ... the importance of it, the Ministry of Education and Training (MOET) has recently decided that English is a compulsory subject in most high schools and universities This decision requires both... whole By analyzing the mistakes made on a given test, the teacher can determine where to concentrate extra class drills and how best to assist each student At the same time, testing enables the teacher... measures what is supposed to measure A listening test with written multiple choice options may lack validity if the printed choices are so difficult to read that the exam actually measures reading