THIẾT kế một bài THI ĐÁNH GIÁ kết QUẢ học tập môn TIẾNG ANH

DECLARATION I certify this minor thesis of the Study Project entitled: Designing an English achievement test for the first - year non-English major students in Son La Teachers’ Training College To total fulfillment of the requirement for the degree Master of Arts. Son La, February 2009 Nguyễn Thị Ngọc Thuý 1 ACKNOWLEDGEMENTS To carry out this MA coursework I am indebted to many people for their encouragement, cooperation, and advice. First and foremost, I would like to express my deepest gratitude to Mr. Vũ Văn Phúc, my supervisor for his useful advice, insightful ideas, and dutiful supervision. I also would like to take this opportunity to express my thanks to all the colleagues in the English department in STTC (Son La Teachers’ Training College), for their help in answering questions in surveys, direct interviews, for their constructive suggestions about this research. I would like to give my special thanks to the students of the first year at STTC who have actively participated in doing the sample test, the surveys and responding to my interviews. Last but not least, my sincere thanks go to my family, my classmates, my friends, especially my husband who encouraged and helped me to carry out the thesis. 2 ABSTRACT Testing plays a very important role in teaching process which helps teachers to assess their teaching as well as their students’ learning. Evaluating a test in terms of its qualities such as reliability and validity is very necessary as to ensure the usefulness of this assessment instrument. However, this issue receives little considerations from teachers at Son La Teachers’ Training College. Thus, this minor study was designed to evaluate two qualities, reliability and validity, of the achievement test for the first year non-major English students at Son La Teachers’ Training College. The MA thesis entitled “Designing an English achievement test for the first - year non-English major students in Son La Teachers’ Training College” deals with the problems of the current testing situation in the College. On doing the study the author attempts to study the possible reasons for the low exam results of the students in the English program and to propose an elementary achievement test for the first year non-major English students. The study is based on both the analysis of the English course including the subjects, the syllabus and the sample test and the analysis and interpretation of the test scores and test items of the suggested test. 3 LIST OF ABBREVIATION 1 STTC Son La Teachers’ Training College 2 FV Facility Value 3 Sd Standard deviation 4 p Level of difficulty 5 D Discrimination 6 g-d Good discrimination 7 m-d Medium discrimination 8 b-d Bad discrimination 9 b-i Bad item 10 α Coefficient alpha 11 IELTS International English Language Testing System 12 TOEFL Testing English as a Foreign Language LIST OF TABLES AND FIGURES TABLE Table 1 Test item types for Reading and Writing skills and Grammar, Vocabulary 4 Table 2 Time frame and the units Table 3 The key points of those 8 units Table 4 The specification grids of the test Table 5 Frequency distribution in the final achievement test Table 6 The standard derivations of 5 scales in the test Table 7 The interpretation of the item difficulty of the test Table 8 The result of item discrimination of the test Table 9 The coefficient alphas of the 5 scales in the test FIGURES Figure 3.3.4.1 Histogram of score distribution TABLE OF CONTENTS Page Declaration i Acknowledgement ii Abstract iii List of abbreviation iv List of tables and figures v Tables of contents vi CHAPTER 1: INTRODUCTION 1 1.1 Rationale 1 1.2 Scope of the study 3 5 1.3 Aims of the study 3 1.4 Methods of the study 4 1.5 Research questions 4 1.6 Design of the study 4 CHAPTER 2: LITERATURE REVIEW 5 2.1 Basic concepts of testing 5 2.2 Types of tests 5 2.2.1 Proficiency tests 5 2.2.2 Achievement tests. 6 2.2.3 Diagnostic tests 8 2.2.4 Placement tests 8 2.2.5 Progress tests 8 2.2.6 Direct versus indirect tests 9 2.2.7 Discrete point verse integrative testing 10 2.2.8 Norm – referenced versus criterion – referenced testing 11 2.2.9 Objective testing versus subjective testing 11 2.2.10 Communicative language testing 12 2.3 Characteristics of a good test 13 2.3.1 Validity 13 2.3.1.1 Construct validity 13 2.3.1.2 Content validity 13 2.3.1.3 Face validity 14 2.3.1.4 Backwash validity 14 2.3.1.5 Criterion-related validity 15 2.3.2 Reliability. 15 2.3.3 Discrimination 15 2.3.4 Practicability. 16 6 2.4 Test items for reading skill, writing skill, grammar, and vocabulary 17 2.4.1 Test items 17 2.4.2 Language components and language skills. 17 2.4.3 The test item types used to evaluate languages components and language skills. 18 CHAPTER 3: THE STUDY 19 3.1 The subjects and the current English teaching, learning and testing situations at STTC 19 3.1.1 Students and their backgrounds 19 3.1.2 The English teaching staff 19 3.1.3 English teaching and learning at STTC 20 3.1.4 The objectives of the elementary English course 21 3.1.5 Teaching material used for first-year students at STTC 22 3.2 The current testing situation at STTC 24 3.3 The proposed construction of the achievement test for the first year students at STTC 25 3.3.1 Test objectives 25 3.3.2 The Paper Specification Grids for the 2 nd Term Achievement Test 25 3.3.3 Data collection 26 3.3.4 Interpretation and test score analysis: 27 3.3.4.1 The frequency distribution 27 3.3.4.2 The central tendency 27 3.3.4.3 The dispersion 28 3.3.5 Test items evaluation 28 3.3.5.1 The item difficulty 28 3.3.5.2 The item discrimination 30 3.3.6 Estimating the reliability of the test 32 3.4 Teachers’ and students’ comments 32 CHAPTER 4: CONCLUSION 34 4.1 Summary of the study 34 4.2 Limitation 34 4.3 Suggestion for further study 35 REFERENCES -1- APPENDICES -3- Appendix 1: The Sample of The 2 nd term Achievement Test -3- Appendix 2: The result of test analysis using ITEMAN software -8- 7 CHAPTER 1: INTRODUCTION 1.1 RATIONALE The importance of language testing is recognized by virtually all professionals in the field of language education. It is of special importance in educational system that is highly competitive as testing is not only an indirect stimulus to learning, but also plays a crucial role in determining the success or failure of an individual's career with direct implications for his future earning power. "Thus, testing is an important tool in educational research and for programme evaluation, and may even throw light on both the nature of language proficiency and language learning"(Lauwerys and Seanlon, 1969). Likewise, in the teaching and learning foreign language process, testing takes a very important role. Language testing is one of the most important ways to evaluate how students acquire when they learn a foreign language. Through tests teachers know not only the success or failure of learners but also how well the learners use what they have been taught. Moreover, the learners know what they gain, what they can apply, and what they cannot. Moore (1992, p.138) states: “Evaluation is an essential tool for teachers because it gives them feedback concerning what the students have learned and indicates what should be done next in the learning process. Evaluation helps you to better understand students, their abilities, interests, attitudes and needs in order to better teach and motivate them.” Nga (1997, p.1) reaches the same conclusion: “Tests are assumed to be powerful determiners-of what happens in classroom and it is commonly claimed that they affect teaching and learning activities both directly and indirectly.” Therefore, testing is an important part of the teaching and learning process; but has it been given adequate attention and careful study yet? Test researchers (Hughes, 1989; Brown, 1995; Read, 1982; Hai, 1999; Tuyet, 1999) in general claim that unfortunately tests have got a bad rap in recent years and not without reason. More often than not, tests are seen by learners “ as dark clouds hanging over their heads, upsetting them with thunderous anxiety as they anticipate the lightning bolts of questions they do not know and worst of all a flood of disappointment if they do not make the grade” (Brown, 1994a: p.373). Hughes (1989, p.1) makes another comment on recent language testing: “It cannot be denied that a great deal of language testing is of very poor quality. Too often language tests have a harmful effect on teaching and learning and too often they fail to measure accurately whatevaer it is 8 they are intended to measure.” This coupled with the fact that teachers frequently lack formal training in educational measurement techniques and they tend to be alienated from the testing process. They regard it as a necessary evil, an intrusion on their regular instructional activities. At present, English tests at Son La Teachers’ Training College (STTC) have the following characteristics: - It has not been given appropriate attention and careful study - Its role in teaching and learning has not been fully recognized. - Almost language teachers think that teachers should be responsible for making tests because testing is one part of teaching and learning activities that students have to pass. - There has been a tendency using commercial (ready-made) tests rather than teacher self- made tests since commercial tests are very convenient and do not take much time to construct. Thus these selected tests may not be relevant to the objectives of the course. - Test content is sometimes found to be unrelated to the objectives of the course and very often many test items in some tests have not been dealt with classes. - Students have complained that there is still a big gap between what is taught and what is tested. An instance for this would be the case when some tests designed for pre- intermediate level are given to students of elementary level. They are so difficult that only few students can accomplish. Therefore, such tests are not valid and reliable. - Using tests exclusively for grading, there is no feedback about the tests. - There has been no discarding of bad tests or bad items. Some items are found to be so difficult that few testees could do whereas there are test items, which are so easy that all testees can obtain the correct answers. Such items should be discarded or replaced. - Moreover, due to the fact that the writing and reading comprehension tests at the university are totally designed with multiple choice techniques so students can easily cheat by asking and copying answers from their classmates. - Apart from those carefully designed tests, some others are still of low and poor quality and these do not accurately measure the students' real ability. Perhaps the test writer only pays attention to the fulfillment of his/her duty, which is to give tests, rather than to the effectiveness of the tests. Those tests often fail to measure accurately whatever they are intended to measure. 9 - Finally, the last testing problem at STTC is that some of the tests may lack reliability because they are not pre-tested anywhere else for the sake of confidentiality. Truly, for the sake of "confidentiality" test designers are often informed to write tests at short notice, just some time before it is administered. In such circumstances who can say for sure that the required standards, criteria will be met by the test writers? Therefore, a well-design test is necessary for every language level especially for college level since it is the elementary level, which aims at acquiring survival English and diagnosing students’ aptitudes in the course and what they have to study to improve both their knowledge and skills. In this minor thesis, the author bases herself on the knowledge of testing and testing situation to propose a sample achievement test for the first year students who have been taught the student’s book New Headway English Course (elementary level) from unit 1 to unit 8. 1.2 SCOPE OF THE STUDY The scope of the study focuses on the existing situations at Son La Teacher’s Training College. I design a sample test only on writing and reading skills focusing on grammar, vocabulary, reading and writing skills. The study provides investigated and analyzed data of the achievement test for the first-year non-English major students. Moreover, the teachers’ and students’ comments on the test and their suggestion for its improvement will be presented in this thesis. 1.3 AIMS OF THE STUDY The aim of the study is to report a research examining the current testing situations and language tests for non-English majors at STTC with great emphasis on analyzing the result of the sample test, the teachers’ and students’ comments on the test and their suggestion for its improvement. The specific aims of the study are: 1. To investigate the STTC teachers’ evaluation and students’ evaluation of the sample test concerning its content, time allowance and its format. 2. To investigate the teachers’ suggestions and students’ suggestions for improving testing situations and language tests at STTC. 10 [...]... a particular aspect of a particular language.” Harrison (1983) remarks that this kind of tests is used at the ends of a unit in the course book or after a lesson designed to teach one particular point This kind of test is reasonably straight-forward to find out what skills are applied well or badly by the learners Otherwise, this leads to disadvantage, as it is not so easy to obtain a detailed analysis... the agreement between markers by establishing, and maintaining adherence to, explicit guidelines for the conduct of this marking The third aspect of reliability is that of parallelforms of a test to be devised The concept of reliability is particularly important when language tests within the communicative paradigm one considered Moreover, Davies (1968) stresses that reliability is the first essential... best option from four given options This part accounts for 30 % of the total mark Section II asks students to match questions with suitable answers The objective is to check students’ grammar structures and vocabulary This part accounts for 20 % of the total mark 33 Section III checks students’ writing skill through building five correct sentences using given words This part accounts for 10% of the... curriculum within a particular time frame.” Another useful comment on achievement tests offered by Finocchiaro and Sako (1983:15) is that achievement types or attainment tests are widely employed in any language teaching institutions They are used to measure the amount of degree of control of discrete language and cultural items and of integrated language skills acquired by the students within a specific... detailed analysis of a learner’s command of grammatical structures In order to be sure of this, we would need a number of examples of the choice the student made between the two structures in every different context on which we thought was significantly different and important enough to warrant obtaining information Tests of this kind still need a tremendous amount of work to produce Whether or not they become... done this successfully He also indicates several attractions of direct testing Firstly, if teachers want to assess pupils’ ability, it is relatively straightforward to create the conditions, which will elicit the behavior based on judgments Secondly, in his opinion at least in the case of the productive skills, the assessment and interpretation of students’ performance is quite straight - forward Thirdly,... task dependency, with tasks in one section of the test building upon the content of earlier sections, including the test taker's answers to those sections Third, communicative tests can be characterized by their integration of test tasks and content within a given domain of discourse Finally, communicative tests attempt to measure a much broader range of language abilities including knowledge of cohesion,... must be related to the courses with which they are concerned, but the nature of this relationship is still a matter of disagreement amongst language testers According to some testing experts, the content of a final achievement test should be based directly on a detailed course syllabus or on the books and other material used This has been referred to as the syllabus–content approach It has an obvious... instructions (open ended questions) - Reordering - Context-based 26 CHAPTER 3: THE STUDY This chapter provides information about the current teaching, learning and testing English situations at STTC as the setting of the study The analyzed data from the sample test result and survey questionnaire is provided in this chapter 3.1 THE SUBJECTS AND THE CURRENT ENGLISH TEACHING, LEARNING AND TESTING SITUATIONS... syllabus–content approach It has an obvious appearance, since the test only contains what it is thought that the students have actually encountered, and thus can be considered, in this respect at least, a fair test The disadvantage of this type is that if the syllabus is badly designed, or the books and other materials are badly chosen, then the results of a test can be very misleading Successful performance . usefulness of this assessment instrument. However, this issue receives little considerations from teachers at Son La Teachers’ Training College. Thus, this minor. actually encountered, and thus can be considered, in this respect at least, a fair test. The disadvantage of this type is that if the syllabus is badly designed,