Disigning an english achievement test for the 12th form of non english major at le quy don high school dong da, hanoi

1 VIETNAM NATIONAL UNVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES FACULTY OF POST-GRADUATE STUDIES VŨ HOÀNG DUNG DESIGNING AN ENGLISH ACHIEVEMENT TEST FOR TH THE 12 FORM OF NON-ENGLISH MAJORS AT LE QUY DON HIGH SCHOOL – DONG DA, HANOI (Thiết kế kiểm tra Tiếng Anh cuối kì cho học sinh lớp 12 khơng chun ngữ trường Trung học phổ thông Lê Quý Đôn – Đống Đa, Hà Nội) M.A Minor Programme Thesis Field: Methodology Code: 60 14 10 Hanoi, August 2010 VIETNAM NATIONAL UNVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES FACULTY OF POST-GRADUATE STUDIES VŨ HOÀNG DUNG DESIGNING AN ENGLISH ACHIEVEMENT TEST FOR TH THE 12 FORM OF NON-ENGLISH MAJORS AT LE QUY DON HIGH SCHOOL – DONG DA, HANOI (Thiết kế kiểm tra Tiếng Anh cuối kì cho học sinh lớp 12 không chuyên ngữ trường Trung học phổ thông Lê Quý Đôn – Đống Đa, Hà Nội) M.A Minor Programme Thesis Field: Methodology Code: 60 14 10 Supervisor: Pham Thi Hanh, M.A Hanoi, August 2010 TABLE OF CONTENTS STATEMENT OF AUTHORSHIP ACKOWLEDGEMENTS TABLE OF CONTENTS ABSTRACT LIST OF ABBREVIATIONS LIST OF TABLES AND APPENDICES ………………………………………… viii Part A: INTRODUCTION………………………………………………………… 1 ….1 Rationale for the study………………………………………………………… Aims and scope of the study………………………………………………… …… Methods of the study………………………………………………………… …….2 Research questions…………………………………………………………… … … Design of the study…………………………………………………………… Part B: LITERATURE REVIEW……………………………………………….… Basic Concepts of Testing…… …………………………………………………….4 Roles of testing……………… …………………………………………………… Types of Test………………………………………………… …………………… 3.1 Achievement tests…………………………… 3.2 Placement tests……………………………… 3.3 Diagnostic tests……………………………………………… ……………… 3.4 Proficiency tests…………………………… 3.5 Direct versus indirect testing………………… 3.6 Discrete point versus integrative testing…… 3.7 Norm-referenced versus criterion-referenced 3.8 Objective versus subjective testing………… 3.9 Communicative language testing…………… Characteristics of a Good Test……………………………………… …………… 14 4.1 Validity……………………………………………………… ……………… 14 4.1.1 Face validity………………………………………… ……………… 14 4.1.2 Content validity……………………………………… ……………… 15 4.1.3 Construct validity……………………………………… …………… 16 4.2 Reliability……………………………………………………… …………… 17 4.3 Practicality…………………… …………………………………………… 18 4.4 Discrimination…………… ………………………………………………… 19 Test Items……………………… ………………………………………………… 19 5.1 Direct test items…………… ……………………………………………… 19 5.2 Indirect test items………… ………………………………………………… 20 Part C: THE STUDY…………………………………….………………………… 22 The issues influencing current testing situation at LQD-DD High School…… … 1.1.The students and their backgrounds…… 1.2.The English teaching staff……………… 1.3.English teaching and learning………… 1.4.The objectives of the K-12 course……… 1.5.Teaching material used for the K-12…… 1.6.The content and construction required f The current testing situation………………………………………………… …… 26 The process of testing………………………………………………… ………… 27 3.1 Determining the purpose of the test……………………………… ……… 28 3.2 Planning the test……………………………………………………… …… 28 3.3 Selecting items and tasks…………………………………………… ……… 29 3.4 Administering the test……………………………………………… ……… 29 3.5 Scoring and rating………………………………………………… ……… 30 3.6 Descriptive data from test score…………………………………… ……… 30 3.6.1 Frequency distribution 31 3.6.2 Measures of central tendency 32 3.6.3 Measures of dispersion 33 3.6.4 Item analysis 34 Survey 36 4.1 Data collection procedure 36 4.2 Findings 37 Archiving ………………………………………………… ……………………… 38 Some discussions from test result analysis 38 Part D: CONCLUSION ……………….…………………………………………… 39 Conclusion ………………………………………………………………………… 39 Recommendation …………………………………………………………………… 39 REFERENCES……………………………………………………………………… 40 LIST OF ABBREVIATIONS D: FV: GCSE: K-12: L: LQD – DD: MCQs: S.D.: U: LIST OF TABLES AND APPENDICES Table 1: Objectives and contents of the test Table 2: The frequency of marking scores Table 3: The measures of central tendency Table 4: The measures of dispersion Table 5: Facility value and Discrimination index APPENDIX 1: The contents of the eight units taught in the term of K-12 APPENDIX 2: The structure of English tests for the GCSE national examination APPENDIX 3: Revision outline for the term test APPENDIX 4: The -term English examination for K-12 APPENDIX 5: Keys for the -term English examination for K-12 APPENDIX 6: Students’ performance on test arranged from the highest score to the st st st st lowest score APPENDIX 7: The survey questionnaire for students APPENDIX 8: The survey questionnaire for teachers 10 Part A: INTRODUCTION Rationale for the study Assessment plays a significant part in the field of linguistics It allows teachers as well as administrators to make important decisions regarding the proficiency, placement, and achievement of second language learners Among some kinds of assessment, testing is the one that is typically used at the end of a stage of instruction to measure students‟ achievement Henning (1987, p 1) says that the most common purpose of language tests is “to pinpoint strengths and weaknesses in the learned abilities of the student” Heaton (1990, p 9) also shares the idea that it is “to find out how well the students have mastered the language areas and skills which have just been taught” Besides, teachers can use language tests to encourage students Students can show their ability and progress through tests Heaton (1990, p 10) agrees that tests are also “used for the purpose of increasing motivation” Having good results, students will be eager for their coming study With not very good test results, they will know which level of language they are at so as to set a more realistic goal for themselves Being aware of the above things, the author of this study, who has been a teacher of English for about 10 years and has written many tests to evaluate the language abilities of students, has noticed that not all the tests have satisfying results Some tests have bad results, which means that too many students get bad marks or too many students get very good marks Teachers cannot use the results of these tests to categorize the students Therefore, such tests are not reliable Taking the problem into consideration, the author has found out that it is the technique of designing tests that leads to the situation When writing tests, the test writer may only feel that this or that test item is suitable for their students She or he may have never analyze the test designing and the test result carefully in order to find out the causes for the problem On the other hand, a well-designed test is always necessary for all students of every language level, especially those of high school level - the elementary level which aims at acquiring survival English Therefore, getting more knowledge and experience in designing tests is the author‟s desire In this minor thesis, the author would like to base herself on the knowledge of th testing and testing situation to design an achievement test for the 12 form of non-English majors at Le Quy Don-Dong Da High School, Ha Noi 11 Aims and scope of the study This study seeks to use the academic knowledge in test design to write a reliable th achievement test for the 12 form of non-English majors at Le Quy Don-Dong Da High School Because this is an achievement test given to students after a study period of about st four months, the term of the school year, and students are required to use what has recently been taught and practiced to the test, we, the teachers of English at LQD – DD High School, expect our students to score fairly high marks, at least 70% of which should be from mark upwards Therefore, this study specifically seeks to answer such questions as whether the items of the test meet the requirements of the course, whether these test items are suitable for at least 70% of students to get marks from upwards and what recommendations should be given to improve the testing situation so that we can build a bank of reliable tests for the school in the coming years As students of grade 12 are going to take part in the national examination for General Certificate of Secondary Education (GCSE), they should be given to a chance to get used to the structure of this kind of examination Therefore, the study will focus on the existing situation at LQD – DD High School, which means that the author will design a multiple choice written achievement test only on such fields as phonetics, grammar, vocabulary, communicative function, reading and writing skills as directed in the book called “Cấu trúc đề thi môn Ngữ văn , Lịch sử , Điạ lí, Ngoại ngữ” (i.e “The Test Structures of Literature, History, Geography, Foreign Languages”) by Nguyen (2009, p 104) The study will also provide analyzed data of the test, the teachers‟ and students‟ comments on the test as well as their suggestions for its improvement Methods of the study The main methods employed in this study included test design, test result analysis and teacher and student survey For the purpose of designing tests, the author based on the theory and principle of language testing, major characteristics of a good test, especially a good achievement test and testing situation at LQD-DD High School The test result was analyzed by the software SPSS to present the frequency distribution, central tendency and dispersion of the test score The analysis of test items is carried out manually Finally, the survey enabled the author to collect teachers‟ and students‟ opinions on the reliability and suitability of the test A questionnaire in both English and Vietnamese their contribution to mankind The awards are usually presented in Stockholm on December 10, with the King of Sweden officiating, an appropriate tribute to 56 Alfred Nobel on the anniversary of his death Each prize includes a gold medal, a diploma, and a cash award of about one million dollars 41.What does this passage mainly discuss? A Alfred Bernard Nobel C Great contributions to mankind 42.How often are the Nobel prizes awarded? A Five times a year C Once every two years 43 A Nobel prize would NOT be given to _ A an author who wrote a novel B a doctor who discovered a vaccine C a composer who wrote a symphony D a diplomat who negotiated a peace settlement 44 Why were the prizes named for Alfred Bernard Nobel? A He left money in his will to establish a fund for the prizes B He won the first Nobel prize for his work in philanthropy C He is now living in Sweden D prizes He serves as a chairman of the committee to choose the recipients of the 45 Why are the awards presented on December 10? A Because it is a tribute to the King of Sweden B Because Alfred Bernard Nobel died on that day C Because that date was established in Alfred Nobel‟s will D Because the Central Bank of Sweden administers the trust VII Choose the best option from the four sentences which are made up from the cues given 46 Family rules / help / raise children / run a household / easier and smoother A Family rules help to raise children but run a household easier and smoother B Family rules help raise children or run a household more easier and smoother C Family rules that help raise children and run a household easier and smoother D 47 Family rules help raise children and run a household easier and smoother He / not / blame / accident A He wasn‟t the blame of the accident B He wasn‟t to blaming himself for the accident C He wasn‟t to be blamed the accident 57 D He wasn‟t to blame for the accident 48 By / age / 20, Makengie / write / four / bestseller A By the age 20, Makengie could write four bestsellers B By the age 20, Makengie had written four bestsellers C By the age 20, Makengie was writing write four bestsellers D By the age 20, Makengie has been writing four bestsellers 49 Teachers / said / have / crucial / role / educational system A The educational system are said to have crucial role in teachers B The teachers said to have crucial role in education system C Educational teachers are said to have crucial role in the system D Teachers are said to have crucial role in the educational system rd 50 We / decide / give / house-warming / party / May rd A We have decided to give a house-warming party on May B We have decided giving a house-warming party on May C We decided to give a house-warming party in May D We decided giving a house-warming party on May rd rd rd 58 APPENDIX st Keys for the -term achievement test for K-12 For each correct answer students get points Total: 100 points A D B B A C C D A 10 A 59 59 APPENDIX Students’ performance on test arranged from the highest score to the lowest score + refers to correct answer The letters refer to the distractors selected by students Item 1-5 Student 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 +++++ +++++ +C+++ D++B+ D++++ BCC++ C++C+ BCC++ +++++ +++++ +++++ ++D++ +++++ D+D++ CAB+C BA+B+ ++BDC +++++ ++CD+ +AB++ ++D++ B+BD+ +++++ D+BDB CACA+ +++++ ++B++ B+BD+ DD++C 6–10 +++++ ++++A +++++ +++++ ++++D ++++B +++++ ++++B +C+++ +C+++ +++++ +++++ ++B++ ++++B +++++ +++++ +++++ ++AB+ AC+++ +CA++ ++B+B ++++B ++BCD BC++B +++++ +AA++ +C+++ A+++B ++++B 60 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 B++DB ++C++ B+BD+ +AC++ D+C++ C+B++ ++BD+ ++++B ++CD+ +BB++ +++++ +++++ C+C+B D+C++ B+B++ +CCD+ CAD+D +ABD+ ++++D +ABA+ DCBA+ +A+++ +++++ A+++B +C+++ ++B++ +AA++ ++++B +++C+ +A++B B++++ CC+++ +DA++ ADB+D +A+CC ++++C +++++ C+A+B +DD+C +AA+B CA++C +D+C+ + + + + A A B D + + B B + D + + A + A A D 61 APPENDIX THE SURVEY QUESTIONNAIRE BẢNG CÂU HỎI KHẢO SÁT th (For students of the 12 grade at Le Quy Don-Dong Da High School) (Dành cho học sinh khối 12 trường THPT Lê Quý Đôn- Đống Đa) This questionnaire is to find out how you feel about “The st Term English th Achievement Test” for students of the 12 grade at Le Quy Don-Dong Da High School Your responses are important, as they will enable improvement on test design All the information you provide will be kept confidentially Thank you very much for your valuable contribution! Bảng câu hỏi nhằm khảo sát xem em nghĩ về “Bài kiểm tra Tiếng Anh cuối học kì 1” vừa qua dành cho học sinh khối 12 Những phản hồi em rất quan trọng chúng giúp nâng cao chất lượng kiểm tra sau Tất thông tin em cung cấp đều giữ bí mật Rất cám ơn đóng góp q báu em Please complete the survey and return it within a period of 45 minutes Hãy hoàn thành bảng câu hỏi nộp lại thời gian tiết hocc̣ st “The Term English Achievement Test” has been constructed as follows: (based on “Cấu trúc đề thi môn Ngữ Văn, Lịch Sử, Địa lí, Ngoại Ngữ” – Education Publishing House, 2009) “Bài kiểm tra Tiếng Anh cuối học kì 1” thiết kế dựa theo yêu cầu “Cấu trúc đề thi môn Ngữ Văn, Lịch Sử, Địa lí, Ngoại Ngữ” Nhà Xuất Bản Giáo Dục năm 2009 62 Time allowance: 60 minutes Thời gian làm bài: 60 phút Lĩnh vực Ngữ âm Ngữ pháp – Từ vựng Chức giao tiếp Kĩ đọc Kĩ viết Please give some information by both ticking a box and writing down in the space in each question Hãy cung cấp thông tin cách tick vào ô trống viết vào phần để trống câu hỏi Does the test measure what you have been taught? Bài kiểm tra có kiểm tra em học không? Tại sao? A Yes Why? ……………………………………………………………………………………… In your opinion, which test items best measure your real ability? (you can choose more than one) Theo em lĩnh vực kiểm tra đánh giá xác nhất khả thực em? (em chọn lựa chọn) A Phonetics (sentences: 1-5) Ngữ âm B Grammar and vocabulary (sentences: 6,7,10-13, 16,17,19-29) Ngữ pháp và từ vựng C Communicative function (sentences: 8,9,14,15,18) 63 Chức giao tiếp D Reading skill (sentences: 36-45) Kĩ đọc E Writing skill: (sentences: 31-35, 46-50) Kĩ viết As for you, this test can measure your … (you can choose more than one) Theo em, kiểm tra đánh giá …của em (em cóthểchoṇ mơṭlưạ choṇ) A Speaking skill (kĩ nói) Can the phonetics part (sentences 1-5) measure your right way to pronounce? A Yes Have you ever practiced such words/phrases with communicative function as in sentences 8, 9, 14, 15, 18? Em luyện tập dùng từ/ngữ thể hiện chức giao tiếp câu 8, 9, 14, 15, 18 chưa? A Yes B No As for the part of nouns, pronouns, verbs, adjectives, connectors, etc… (sentences 22-30), does it contain only forms and uses you have practiced? Phần danh từ, đại từ, động từ, tính từ, từ nối, v.v…(từ câu 22 đến câu 30) có bao gồm dạng cách dùng em luyện tập không? A Yes Are you familiar with the sentence structures in sentences 16, 17, 19-21? Em có thấy quen với cấu trúc câu câu 16, 17, 19-21 không? A Yes Have you ever practiced such word formations as in sentences 11-13 and such fixed words/phrases as in sentences 6, 7, 10? Em thực hành dạng từ câu 11 đến 13 từ/cụm từ cố định câu 6, 7, 10 chưa? A Yes B No 64 As for you, the structures and words given in the writing parts (part IV and part VII) are: Theo em, cấu trúc từ cung cấp phần viết (phần IV VII): A related to what you have been taught liên quan đến kiến thức em học B partly related to what you have been taught liên quan phần đến kiến thức em học C not related to what you have been taught không liên quan đến kiến thức em học 10 In your opinion, the topics of the reading texts are: Theo em, chủ đề hai đọc là: A familiar quen thuộc 11 In your opinion, the reading texts (part V and part VI of the test) are: Theo em, hai đọc (phần V VI kiểm tra): A very long and difficult dài và khó B rather long and difficult dài và khó C reasonable hợp lí D easy dễ E very easy dễ 12 Are the instructions clear enough for you to understand? Yêu cầu đầu có rõ ràng khơng? A Yes B No 13 Are there any errors in the test? Em có phát hiện lỗi kiểm tra không? A Yes 14 B No According to you, time allowance for this test is … Theo em, thời gian cho phép để làm kiểm tra … A too much 65 nhiều B enough đủ C not enough không đủ D If too much or not enough, how much time you think is reasonable for this test? Nếu thời gian nhiều không đủ, theo em thời gian để làm kiểm tra hợp lí? ……………………………………………… 15 Please give comments or suggestions towards the improvement of the existing test Hãy vui lòng đưa nhận xét gợi ý để kiểm tra hoàn thiện ……………………………………………………………………………………………… ………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… THANK YOU VERY MUCH FOR YOUR GREAT CONTRIBUTION! 66 APPENDIX THE SURVEY QUESTIONNAIRE (For teachers of English at Le Quy Don-Dong Da High School) This questionnaire is to ask for your comments and ideas about the design of the recent English Achievement Test for students of the 12 th grade at Le Quy Don-Dong Da High School in terms of its content and time allowance Your assistance in completing the questionnaire is highly appreciated, as it will enable improvement on test design All the information you provide is solely used for the study purpose and you can be confident that you will not be identified in any discussion of the data Please complete the questionnaire and return it by December 25, 2009 st “The Term English Achievement Test” has been constructed as follows: (based on “Cấu trúc đề thi mơn Ngữ Văn, Lịch Sử, Địa lí, Ngoại Ngữ” – Education Publishing House, 2009) Time allowance: 60 minutes Main skills & language components Phonetics Grammar & Vocabulary Communicative function Reading ... and testing situation to design an achievement test for the 12 form of non -English majors at Le Quy Don- Dong Da High School, Ha Noi 11 Aims and scope of the study This study seeks to use the. .. scheduled to be performed after another one The school drum was used to signal the beginning and the end of the test performance When there was only 15 minutes left for the test performance, all the. .. academic knowledge in test design to write a reliable th achievement test for the 12 form of non -English majors at Le Quy Don- Dong Da High School Because this is an achievement test given to