An investigation into teachers perceptions of language test validity and their practice in designing tests at some upper secondary schools in quang tri

MINISTRY OF EDUCATION AND TRAINING HUE UNIVERSITY COLLEGE OF FOREIGN LANGUAGES - HOANG THI HUONG AN INVESTIGATION INTO TEACHERS' PERCEPTIONS OF LANGUAGE TEST VALIDITY AND THEIR PRACTICE IN DESIGNING TESTS AT SOME UPPER-SECONDARY SCHOOLS IN QUANG TRI MA THESIS IN THEORY AND METHODOLOGY OF ENGLISH LANGUAGE TEACHING HUE, 2014 MINISTRY OF EDUCATION AND TRAINING HUE UNIVERSITY COLLEGE OF FOREIGN LANGUAGES - HOANG THI HUONG AN INVESTIGATION INTO TEACHERS' PERCEPTIONS OF LANGUAGE TEST VALIDITY AND THEIR PRACTICE IN DESIGNING TESTS AT SOME UPPER-SECONDARY SCHOOLS IN QUANG TRI MA THESIS IN THEORY AND METHODOLOGY OF ENGLISH LANGUAGE TEACHING CODE: 60.14.01.11 SUPERVISOR: TRUONG BACH LE, D.E.d HUE, 2014 i BỘ GIÁO DỤC VÀ ĐÀO TẠO ĐẠI HỌC HUẾ TRƯỜNG ĐẠI HỌC NGOẠI NGỮ *** - HỒNG THỊ HƯƠNG TÌM HIỂU NHẬN THỨC CỦA GIÁO VIÊN VỀ TÍNH GIÁ TRỊ CỦA BÀI KIỂM TRA NGƠN NGỮ VÀ THỰC TẾ THIẾT KẾ ĐỀ KIỂM TRA CỦA GIÁO VIÊN Ở MỘT SỐ TRƯỜNG THPT TRÊN ĐỊA BÀN TỈNH QUẢNG TRỊ LUẬN VĂN THẠC SĨ LÝ LUẬN VÀ PHƯƠNG PHÁP DẠY HỌC BỘ MÔN TIẾNG ANH MÃ SỐ: 60.14.01.11 NGƯỜI HƯỚNG DẪN KHOA HỌC : TS TRƯƠNG BẠCH LÊ HUẾ, 2014 ii STATEMENT OF AUTHORSHIP I hereby acknowledge that this study is mine The data and findings discussed in the thesis are true and have not been published elsewhere Author HOANG THI HUONG iii ABSTRACT This study attempts to investigate teachers' perceptions of language test validity and their practice in designing tests at some upper-secondary schools in Quang Tri province A combination of quantitative and qualitative methods, including questionnaire, interviews and test sample evaluation, was adopted to get data for this paper The data analyzed and discussed provide an overview on teachers' perceptions of language testing in general and the validity of language tests as well as basic principles of test design practice In addition, the evaluation of the test samples using criteria in accordance with language test validity helps show how the teachers design English tests in reality The findings indicate that while the teachers have good perceptions of language tests validity and acknowledge the importance of test design practice, their practice of test design is not as good as their perceptions imply The mismatch between the teachers' conceptions and their practice reveals a need for formal training for teachers in the field of language testing and test design practice iv ACKNOWLEDGEMENTS Above all, I would like to thank my supervisor, Dr Truong Bach Le, for his valuable instruction, advice and encouragement throughout this study Without his practical guidance, this research paper would not have been completed I would also like to express my gratitude to my teachers at Hue University of Foreign Languages, who listened and supported my efforts throughout the process It would be a big mistake for me not to thank my colleagues and friends who heartedly participated in completing the questionnaire, answering the interviews and sending me the test samples Their substantial assistance greatly contributed to the accomplishment of this research Last but not least, I would like to thank my family for their understanding and endless support from the very beginning of my studies to the end of this project I am truly blessed to have such considerable encouragement and assistance v TABLE OF CONTENTS Pages SUB COVER PAGE i, ii STATEMENT OF AUTHORSHIP iii ABSTRACT iv ACKNOWLEDGEMENTS v TABLE OF CONTENTS vi LIST OF ABBREVIATIONS ix LIST OF TABLES AND FIGURES x CHAPTER 1: INTRODUCTION 1.1 Theoretical background 1.2 Aims of the Study 1.3 Significance of the study 1.4 Research Questions 1.5 Organization of the thesis CHAPTER 2: LITERATURE REVIEW 2.1 An overview of language testing 2.1.1 The development of language testing 2.1.2 Purposes of language testing 2.1.3 Common types of English language tests 2.1.3.1.1 Diagnostic tests 2.1.3.2 Placement tests 10 2.1.3.3 Achievement tests 12 2.1.4 The designing of language tests 14 2.2 Test validity 17 2.2.1 Content validity and Face validity 18 2.2.2 Construct validity 19 2.2.3 Criterion-related validity 19 2.2.4 Concurrent validity and Predictive validity 19 2.2.5 Response validity 20 2.3 The relationship between teachers' perceptions and practice 20 2.4 The situation of language testing at upper-secondary schools in Quang Tri province 22 2.4.1 The use of language tests 22 vi 2.4.2 The designing of language tests 24 2.5 Chapter summary 25 CHAPTER 3: RESEARCH METHODOLOGY 26 3.1 Participants 26 3.2 Research Instruments 26 3.2.1 Questionnaire 27 3.2.2 Interviews 29 3.2.3 Test samples 30 3.2.4, Test evaluation sheet, curriculum, test matrices 30 3.3 Procedure of data collection and analysis 32 3.3.1 Questionnaire 32 3.3.2 Interviews 33 3.3.3 Test samples 33 3.4 Chapter summary 33 CHAPTER 4: FINDINGS AND DISCUSSION 34 4.1 Teachers' perceptions of language test validity 35 4.1.1 Teachers' beliefs regarding language testing and the use of language tests 35 4.1.2 Teachers' conceptions of language test validity 38 4.2 Teachers' test design practice in accordance with language test validity 43 4.2.1 Teachers' beliefs regarding language test design 43 4.2.2 How teachers design language tests in reality 45 4.2.2.a Results from the questionnaire and the interviews 45 4.2.2.b Results from test sample evaluation 48 4.3 The relationship between teachers' perceptions of language test validity and their practice of test design 60 4.4 Chapter summary 62 CHAPTER 5: CONLUSION AND IMPLICATIONS 64 5.1 Restatement of the Research Aims 64 5.2 Summary of the Findings 64 5.2.1 What are teachers' perceptions of the validity of language tests? 64 5.2.2 How they conduct the designing of language tests in reality? 64 5.2.3 What is the relationship between their perceptions and their practice?65 5.3 Significance of the Study 65 vii 5.4 Implications of the Study 65 5.5 Further Research 66 5.6 Limitations of the Study 66 REFERENCES 68 APPENDICES 72 viii LIST OF ABBREVIATIONS SP: Specification SPSS: Statistics Package for the Social Sciences TLU: Target Language Use MOET: Ministry Of Education and Training ix APPENDIX TEST SAMPLES 99 Trường THPT VĨNH LINH KIỂM TRA TIẾT Họ tên: Thời gian làm bài: 45' Lớp 10 Ngày kiểm tra: /10/2013 Ngày trả bài: /10/2013 Mã đề : 101 Số báo danh: ………… Phòng thi: ……… Mã đề: 101 I Choose the word having underlined part is pronounced differently from the rest in each group a lovely b garden c lunch d sunny a pea b death c ready d peasant a heat b bit c little d hit II Identify one underlined word or phrase that must be changed for the sentence to be correct My sister don’t really like going to school late A B C D We go to visit Hue last summer vacation A B C D He enjoys play guitar every evening A B C D III Read the following passage then choose the best answer to complete each sentence Thomas Edison was born in Milan, Ohio, on February 11th, 1847 His family moved to Port Huron, Michigan, when he was seven years old Surprisingly, he attended school for only two months His mother, a former teacher, taught him a few things, but Thomas Edison was mostly self-educated His natural curiosity led him to start experimenting at a young age Thomas Alva Edison lit up the world with the invention of the electric light Although the electric light seems to be most useful, it was not his only invention He also invented the record player, the motion picture camera and over 1,200 other things Thomas Edison died at the age of 84 on October 18th, 1931, at his estate in West Orange, New Jersey He left numerous inventions that improved the quality of life all over the world The word was mostly self- educated can be replaced by A thought of himself B mostly taught himself C loved school D was born a genius 100 Which is NOT true about Edison? A He invented the electric light B He didn't go to school at all C He was the author of more than 1,200 inventions D He died in New Jersey How old was Edison when he died? A 24 B 84 C 47 D 16 10 What is considered Thomas Edison's most useful invention? A the record player B the electric light C the camera D telephone C In New Jersey D In Michigan 11 Where was Thomas Edison born? A In Ohio B In Huron IV Choose the best answer 12 I was born on 25th January 25th January is my a date of birth b place of birth c present address d surname 13 My sister _ married before 1990 a get b had got c getting d got 14 It was a rainy day, so we decided……… an umbrella a bringing b to bring c bring d brought 15 Could you please stop ………… so much noise? a make b made c to make d making 16 Children enjoy ………… with household tasks a help b to helping c helping d helped 17 Would you like ………… a cup of coffee? – Sure! a to having b has c had d to have c took usually d usually 18 After lunch, we ………….a rest for an hour a take usually b usually take taking 19 I often go to school……… 6:30 a.m everyday a on b at c in d of 20 The light ………… out while we were having tea a was going b went c had gone d has gone 21 The World War II ………….in 1939 and ended in 1945 a begins b begun c began 101 d beginning 22 …………….were you born? I was born in 1995 a Where b When c Why d Who 23 Hi! What's the matter with you, Nam? a Hi, I have a headache b Hello, I am doing my homework c Hi, I am busy d Hello, I am a student 24 My brother always drives to work a My brother always goes to work in my car b My brother always gets to work by car c My brother always repairs cars at work d My brother’s driver is always off work V Choose the correct form of the word in the bracket 1- Quan has a Civic lesson on Monday (education/ educate) 2- I want to learn English because it is an language (internation/international) Marie Curie harboured the dream of a career (scientific/ scientist) My parents are .with my final examination result (disappointing/ disappointed) VI Finish each of the following sentences in such a way that it means exactly the same as the sentence printed before it Quan always walks to school .… ………… .………to school ? We enjoy coming to visit my grand- parents this weekend  We like…………………………………………………….………………… She denied going out for a drink with me  She refused…………………………………………………………………………… We don’t want to meet that man  We avoid………………… ………………………… -THE END- 102 QUANG TRI TOWN HIGH SCHOOL Name:…………………………… ONE PERIOD TEST A Class: 10 Time: 45 minutes I Choose the word (A,B,C, or D) whose underlined part is pronounced differently from the others (0.75 m) A heat B meat C minute D sheep A far B marvelous C star D match A cut B love C father D much II Choose the word (A,B,C, or D) whose main stress is placed differently from the others.(0.25 m) A enjoy B prevent C special D attend GRAMMAR AND VOCABULARY: III Choose the best option A, B, C, or D that best completes each sentences (4 ms) As a brilliant and mature student, Marie haboured the dream of a scientific career A intelligent B ambitious C strong –willed D good The alarm went off when smoke rose A stopped B rang C went wrong D went down Tom doesn’t have much free time He …………………………… A goes to the concert usually B always goes to the concert C occasionally goes to the concert D goes to the concert very often ……………classes does Ba have every day? – Five classes A How much B How many C What kind D What kinds Tim never chats………………the lesson A while B when C during D within 10 John is not a farmer so he never…………………………… A the transplanting B transplanting C does the transplanting D did the transplanting 11 She can …………… four foreign languages A to speak B speaking C spoken D speak 12 Mathematics is an important , which is required in most of examinations for high school students A object B subject C fact D syllabus 13 When Tom arrived at the station, his girlfriend …………….for the city A already leaves B already left C had already left D has already left 14 I am sorry to keep you ……………… A wait B to wait C waiting D waited 15 People increasingly object ………… in public places A to smoke B smoking C to smoking D smoke 103 16 During the rush hour, streets are often crowded……………people and vehicles A for B of C in D with 17 It is no use asking him for help He is never ………… to help anybody A willing B willingly C willingness D will 18 …………Peter leave for London? - Two weeks ago A Why has B When did C Where will D What did 19 Before you came, I ……………… all my homework A finished B have finished C was finishing D had finished 20 He failed ………… his boss to accept his project A persuade B to persuade C persuading D persuaded IV Supply the verb form of the verbs in brackets (1.75 ms) 21 Mary (meet) Peter three weeks ago 22 I look forward to (see) you at the weekend 23 When I …………… (come) to visit him, he……… (go) to work 24 The house was quiet when I got home Everybody (go) to sleep 25 While I was working in the garden, I (hurt) my back 26 Peter decided ……………… (apply) for the job in this company V Supply the correct form of the word in brackets (0.5 m) 27 I not think that he does well all the subjects at school because he can not control his ………………… (concentrate) 28 His mother’s death is the greatest ……… event that he will never get rid of (tragedy) READING COMPREHENSION: VI Read and answer the questions below (2 ms) When Peter as a young boy in school, his favorite class was science He didn’t like history or art, but he was excellent in mathematics In high school, he also studied chemistry and biology He was an intelligent boy, and he put a lot of effort into his homework He always got the best grades in the class, but he was not satisfied with his tests if they were not perfect Peter went to a famous university, but he couldn’t decide if he wanted to major in chemistry or mathematics In his first year as an undergraduate, he discovered a field that allowed him to study both – chemistry engineering He also studied the basics of engineering in his first two years, and in his junior and senior years, he took all the advanced courses During his senior year, he received a job offered from a large chemical company Peter is now a successful chemical engineer and he is finally satisfied with his job 29 What was Peter’s favorite class? 30 Why did Peter get the best grades in his class? 104 31 What did he study in his first two years as an undergraduate? 32 When did he receive his job offer? VII Make questions for the underlined part (0.75 m) 33 Mary has lived with her aunt since her parents passed away 34 My father got home last night after one-week-tour in America ……………………………………………………………… 35 I was having a bath when the phone rang ……………………………………………………………… 105 LE LOI HIGH SCHOOL 45’ TEST No1- ASurname: Class: 12 I.A Read the passage and then decide whether each of statements is true (T) or false (F) (1m) Smoking is not as common in America as in many other countries Generally, Americans smoke less than Europeans and much less than Asians It is a practice that is becoming less and less socially acceptable Smoking is prohibited in many places It is not allowed in any public building, on any public transportation ( including airplane flights within the USA ), in shops, movies theaters , schools, and office buildings The general rule is if you are indoor, then you probably are not allowed to smoke The exceptions are bars, nightclubs, and some restaurants If a restaurant does allow smoking , it will be only an area that is designated for smokers If you are with someone, even outdoors , it is polite to ask if they mind before you start smoking .1 Smoking isn’t allowed in America .2 Asians smoke less than Europeans and Americans .3 You should ask for permission before smoking Anyone traveling by plane in the United States can smoke B Read the passage and choose the correct answer (2m) If you are invited to someone's house in America for dinner, you should bring a gift, such as a bunch of flowers or a box of chocolates If you give your host a wrapped gift, he or she may open it in front of you Opening a present in front of the gift-giver is considered polite It shows that the host is excited about receiving the gift and wants to show his or her appreciation to you immediately Even if the host doesn't like it, he or she will tell a "white lie" and say how much they like the gift to prevent the guest from feeling bad If your host asks you to arrive at a particular time, you should not arrive exactly on time or earlier than the expected time, because this is considered to be potentially inconvenient and therefore rude, as the host may not be ready 1) The polite host will open the in front of the gift-giver a) wrapped gift b) the gift-giver c) bunch of flowers d) bottle of wine 2) The host always make the guests a) show anger b) feel pleasant c) feel bad d) tell a lie 3) What will the hosts if they don't like the present? a) Give it back b) Receive it without smiling c) Receive it with a lie d) Open it in front of the giver 4) You should arrive if you are invited to a dinner at a particular time a) before a particular time b) exactly on time c) late d) early 5) Which of the following is the closest meaning with the word "rude" _ a) impolite b) helpful c) sad d) angry II Circle the word that has the underlined letters pronounced differently from the rest (1m) A pairs B sports C photographs D parks 106 A obliged B determined C believed D kissed A does B moves C loves D finishes A finished B abandoned C arrived D rained III Write the correct form of each word in the following sentences: (1 m) Most of the students are strongly of the school's approach.( support) 2.An arranged is the one in which the parents choose a husband or wife for their child (marry) IV Rewrite these sentences using the words given (2m) 1.The train left the station before I arrived there => When 2.I started learning French six years ago => I have 3.Rosa asked Juliet, “ Have you watched this film?” => Rosa asked Juliet “Please turn off the TV”, I said to Nam => I asked… .……………………………………………………… V Choose the best option to each question below (3m) 1.Pay more attention _ picture and you can find out who is the robber a to b for c at d on 2.- What an attractive hair style you have got, Mary! - _ a Thank you very much I am afraid b You are telling a lie c Thank you for your compliment d I don't like your sayings 3.- What beautiful dress you are wearing! - Thank you That is nice compliment a a / a b the / Ø c Ø / Ø d the / the 4.Peter was .a hurry to go so he did not stop to greet me a in b on c with d over 5.In my company, the director deputy usually the responsibility for organizing meetings and conferences a makes b gets c takes 107 d becomes 6.Since I up smoking , my health has been better a give b gave c have given d had given Some candidates failed the oral examination because they didn't have enough a confide b confident c confidence d confidential I'd like to see that football match because I one this year a haven't seen b don't see c weren't seeing d didn't see 9.When I came to their house, they chess a were playing b are playing c played d have been playing 10 We are not allowed jeans at school a wear b wearing c to wear d to wearing 11.Body language is a potent from of communication a verbal b non-verbal c tongue d oral 12 Last year , he .an international exam on Maths but he failed to won a prize a take b taked c has taken 108 d took APPENDIX TEST MATRICES 109 TEST SPECIFICATIONS 45-MINUTE TEST No1 – 1st SEMESTER GRADE: 10 * Aims: - To check students’ knowledge about what they have learnt in Unit 1, unit and - To drill students’ skills - To explains and corrects students’ mistakes APPLIC KNOWLE COMPREH TOTOPICS ADGE EN-SION TAL TION I Phonetics: (3 questions, 0,25 mark for each correct answer) - The vowel cluster:/a:/ and /^/ - The vowel clusters: /i:/ and /e/ - The vowel cluster: /i/and /i:/ II Reading: (5 questions, 0,25 mark for each correct answer) - Choose the best answers to complete sentences(5 questions) : Scientist- Thomas Edison III Structures and vocabulary: (13 questions, 0,25 marks for each correct answer) - To inf and gerund 13 - Tenses - Communicative structure - Vocabulary - Interrogative pronoun - Transformation 110 IV Find out mistakes (3 questions, 0,25marks for each correct answer) - Simple past tense - Present simple tense - Gerund 1 V- Word form (4 words, 0,5marks for each correct word) scientist/educate/interna tion/disappointing IV Writing: (4 questions, 0,5 mark for each correct answer) - Making question with interrogative pronoun - Rewriting with gerund and infinitive verbs TOTAL SENTENCES TOTAL MARKS PERCENTAGES 4 16 4,0 40 % 2,0 20 % 111 4,0 40 % 32 10,0 100 % MA TRẬN ĐỀ KIỂM TRA MỘT LẦN TIẾT – MÔN TIẾNG ANH 11 (CƠ BẢN ) Thời gian làm bài: 45 phút Nội dung hình thức kiểm NHẬN BIẾT THƠNG tra HIỂU I Đọc hiểu: (trắc nghiệm câu hỏi (mỗi câu (0,25 đ) MCQs) Đọc đoạn văn chọn câu trả câu 0,25 điểm) lời II Ngữ pháp từ vựng - Từ vựng từ Bài 9,10 11 (trong đọc, nói…) - Động từ với giới từ - Đại từ quan hệ dạng lược - Đại từ quan hệ giới từ - Nguyên mẫu có to III Phát âm trọng âm (trắc nghiệm MCQs) Các nguyên âm phụ âm học IV Sửa lỗi sai: (trắc nghiệm MCQs) - Lược mạnh đề quan hệ dùng khứ phân từ - Lược mạnh đề quan hệ dùng nguyên mẫu có to V Chia động từ dùng dạng từ (TL) - Hiện phân từ - Quá khứ phân từ - Nguyên mẫu có to VI Viết lại (TL) - Dùng mệnh đề quan hệ dạng lược - Dùng đại từ quan hệ Tổng điểm: câu (0,25 điểm) câu (0,25 điểm) câu (0,25 điểm) câu (0,25 điểm) câu (0,25 điểm) câu (0,5 điểm) Tổng câu hỏi (0,25 điểm) 0,75 điểm câu (0,25 đ) câu (0,25 điểm) câu (0,25 điểm) câu (mỗi câu 0,25 điểm) câu (0,25 điểm) VẬN DỤNG 2,5 điểm câu (0,25 điểm) câu (0,25 điểm) 0,75 điểm câu (0,25 điểm) câu (0,25 điểm) câu (0,5 điểm) câu (0,5 điểm) câu (0,5 điểm) câu (0,5 điểm) câu (0,5 điểm) câu (0,5 điểm) câu (0,5 điểm) điểm 3,25 điểm 1,75điểm 112 điểm điểm điểm 10 điểm Chú giải: - Câu mức độ giỏi: phần đọc hiểu cần có suy luận trả lời theo cách học sinh, khơng cóp lại từ đọc Phần câu tường thuật, học sinh phải biến đổi nhiều hơn, ví dụ có đại từ tân ngữ tính từ sở hữu - Câu mức độ TB: học sinh trung bình nhìn vào làm Ví dụ: cần nắm bản; từ vựng quen thuộc học, phát âm từ quen thuộc học sách giáo khoa Từ để hỏi trạng từ thường xuyên - Câu mức khá: cao so với cấp độ không đến mức 113