Xây dựng và xác trị bài kiểm tra học kì kĩ năng nói môn tiếng anh cho học sinh lớp 8 tại một trường thcs tại hà nội

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES FACULTY OF ENGLISH LANGUAGE TEACHER EDUCATION GRADUATION PAPER THE DEVELOPMENT AND VALIDATION OF AN ENGLISH SPEAKING ACHIEVEMENT TEST FOR 8TH GRADERS IN A SECONDARY SCHOOL IN HANOI Supervisor: Dương Thu Mai, PhD Student: Nguyễn Vân Anh Course: QH2014.F1.E1 HÀ NỘI – 2018 ĐẠI HỌC QUỐC GIA HÀ NỘI TRƯỜNG ĐẠI HỌC NGOẠI NGỮ KHOA SƯ PHẠM TIẾNG ANH KHÓA LUẬN TỐT NGHIỆP XÂY DỰNG VÀ XÁC TRỊ BÀI KIỂM TRA HỌC KÌ KĨ NĂNG NĨI MƠN TIẾNG ANH CHO HỌC SINH LỚP TẠI MỘT TRƯỜNG THCS TẠI HÀ NỘI Giáo viên hướng dẫn: Dương Thu Mai, PhD Sinh viên: Nguyễn Vân Anh Khóa: QH2014.F1.E1 HÀ NỘI – 2018 ACCEPTANCE PAGE I hereby state that I: Nguyễn Vân Anh, QH2014.F1.E1, being a candidate for the degree of Bachelor of Arts (Fast-track program), accept the requirements of the College relating to the retention and use of Bachelor’s Graduation Paper deposited in the library In terms of these conditions, I agree that the origin of my paper deposited in the library should be accessible for the purposes of study and research, in accordance with the normal conditions established by the librarian for the care, loan or reproduction of the paper Signature Date i ACKNOWLEDGEMENTS I would like to express my sincere gratitude to my supervisor – Ms Duong Thu Mai, PhD – for her constant support for my study Without her valuable advice and suggestions, together with her encouragement and passion, I could not manage to complete this graduation paper I deeply appreciate the testing experts at FELTE, ULIS-VNU, as well as 8th grade teachers and students at Dong Da Secondary School, who have been the enthusiastic respondents in my research I would also like to send my thanks to my lecturers and my classmates for the insightful comments and encouraging words they have given me Last but not least, I owe my gratitude towards my beloved family members, who have been motivating me throughout the writing of this research i ABSTRACT Thanks to the trend of communicative language teaching and the introduction of the pilot English program, the role of speaking and speaking assessment is being highlighted in secondary school English education However, speaking assessment instruments for secondary students are limited and not many of them are of good quality, due to the complexity of test development and validation This empirical study is an attempt to design and evaluate the content validity of an English speaking achievement test for 8th graders based on conceptualized framework of test construction To select the appropriate test content, observation of English 8’s textbook content and teaching practice, as well as teachers’ opinion collected by questionnaire had been conducted A test was constructed at the end of the study It was unveiled that responsive tasks were most appropriate to assess 8th graders’ speaking performance The survey data gathered from testing experts afterwards has shown that the test content is valid in terms of relevance to objectives, duration, instructions and interaction amount The rating scale and some tasks in the test, however, need further revisions to be more relevant to the course contents Key terms: language tests, test development, test evaluation, content validity ii TABLE OF CONTENTS ACKNOWLEDGEMENTS i ABSTRACT ii TABLE OF CONTENTS iii LIST OF FIGURES, TABLES, AND ABBREVIATIONS vi CHAPTER INTRODUCTION 1 Background of the study Aims and objectives of the study Scope of the study 4 Significance of the study Organization of the study CHAPTER LITERATURE REVIEW Key concepts of language assessment 1.1 Language testing 1.2 Developing classroom language tests 13 1.3 Validating a classroom language test 17 1.4 Testing speaking/oral production 20 Review on related studies 27 2.1 Studies on developing and validating speaking tests worldwide 27 2.2 Studies on developing and validating speaking tests in Vietnam 27 CHAPTER METHODOLOGY 30 The instruction and assessment of speaking skill for 8th graders at Dong Da Secondary School 30 1.1 The instruction of speaking skill 30 1.2 The assessment of speaking skill 31 iii Research questions 31 2.1 What are the components of an English speaking achievement test for 8th graders? 31 2.2 To what extent is the test valid in terms of content validity? 31 Sampling 32 Data collection methods 32 4.1 Observation 32 4.2 Survey 33 4.3 Data collection procedure 34 Data analysis methods 35 CHAPTER FINDINGS AND DISCUSSION 37 The development of the speaking achievement test for 8th graders at Dong Da Secondary School 37 1.1 Test rationale 37 1.2 Content selection 38 1.3 Development of specifications (including scoring procedures) and writing of materials 47 The validation of the speaking achievement test for 8th graders at Dong Da Secondary School 49 CHAPTER CONCLUSION 55 Conclusion and implications 55 Limitations of the study 56 Suggestions for further research 57 REFERENCES 58 APPENDICES 63 APPENDIX TEST SPECIFICATIONS 63 iv APPENDIX QUESTIONNAIRE FOR EXPERT INFORMANTS 70 APPENDIX QUESTIONNAIRE FOR TESTING EXPERTS 75 APPENDIX OBSERVATION SCHEME FOR TEXTBOOK 80 APPENDIX OBSERVATION SCHEME FOR ENGLISH LESSON 81 APPENDIX BOOK MAP OF ENGLISH TEXTBOOK 82 v LIST OF FIGURES, TABLES, AND ABBREVIATIONS LIST OF FIGURES Figure Steps to designing an effective test Figure Different components involved in communication Figure Testing experts’ opinions on the difficulty of test tasks LIST OF TABLES Table A framework for describing the speaking construct Table Speaking performance objectives of the new English course Table Teachers’ opinions on the frequency of task types in English lessons Table Teachers’ opinions on the suitability of speaking task types Table Teachers’ opinion on the likelihood of using speaking task types in real-life communicative situations Table Teachers’ opinion on the level of interest of students in different speaking tasks Table Common Reference Level: global scale Table Frequency of speaking tasks recommended by the textbooks to measure each performance objective Table Frequency of speaking tasks used by teacher to measure each objective Table 10 Testing experts’ opinions on the relevance of test tasks towards course content Table 11 Testing experts’ opinions on the relevance of test tasks towards task objectives Table 12 Testing experts’ opinions on the instructions/prompts of test tasks Table 13 Testing experts’ opinions on the types of interaction in test tasks Table 14 Testing experts’ opinions on the duration of each task and the test as a whole vi LIST OF ABBREVIATIONS VSTEP Vietnam Standardized Test of English Proficiency IELTS International English Language Testing System TOEFL Test of English as a Foreign Language DDSS Dong Da Secondary School CEFR Common European Framework ULIS University of Languages and International Studies VNU Vietnam National University vii APPENDIX QUESTIONNAIRE FOR EXPERT INFORMANTS Hello! My name is Van Anh Nguyen, a senior student from the University of Languages and International Studies Thank you for participating in this questionnaire from my Graduation Paper named “The Development and Validation of a Speaking Achievement Test for 8th graders at a Secondary School in Hanoi” The survey aims at studying the comparability of speaking test tasks with the test teachers’ perspective, before sampling some of them in my self-designed speaking test Answering those questions will take you no more than minutes All of your personal information as well as answers will be kept in great confidentiality and will be serving research purposes only PERSONAL INFORMATION Full name: Years of experience: Email address: Phone number: MAIN CONTENT For the following questions, please give ratings as asked by circling the appropriate numbers How often you use these speaking tasks in your English lessons or speaking tests? SPEAKING TASK Never use Repetition Read-aloud Directed response Sentence/Dialogue completion task Picture-cued Translation (of limited stretches of discourse) Question and answer Giving instructions and directions Paraphrasing Interview Role-play Discussion and conversation Games Oral presentation Picture-cued storytelling Retelling a story, news event Translation (of extended prose) Others (please specify): 1 1 1 1 1 1 1 1 1 1 1 70 Use very frequently 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 5 To what extent you think these types of speaking tasks can elicit samples of students ’ ability to demonstrate English sounds, stress patterns, rhythmic structure, and intonation contours in basic compound sentences? SPEAKING TASK Cannot elicit Repetition Read-aloud Directed response Sentence/Dialogue completion task Picture-cued Translation (of limited stretches of discourse) Question and answer Giving instructions and directions Paraphrasing Interview Role-play Discussion and conversation Games Oral presentation Picture-cued storytelling Retelling a story, news event Translation (of extended prose) Others (please specify): 1 1 1 1 1 1 1 1 1 1 1 Elicit very well 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 5 To what extent you think these types of speaking tasks can elicit samples of students’ ability to give instructions/directions in daily interactions? SPEAKING TASK Cannot elicit Repetition Read-aloud Directed response Sentence/Dialogue completion task Picture-cued Translation (of limited stretches of discourse) Question and answer Giving instructions and directions Paraphrasing Interview Role-play 1 1 1 1 1 71 Elicit very well 2 2 2 2 2 3 3 3 3 3 4 4 4 4 4 5 5 5 5 5 Discussion and conversation Games Oral presentation Picture-cued storytelling Retelling a story, news event Translation (of extended prose) Others (please specify): 1 1 1 1 1 2 2 2 2 2 3 3 3 3 3 4 4 4 4 4 5 5 5 5 5 To what extent you think these types of speaking tasks can elicit samples of students’ ability to participate in short and simple exchanges of information about learnt topics? SPEAKING TASK Cannot elicit Repetition Read-aloud Directed response Sentence/Dialogue completion task Picture-cued Translation (of limited stretches of discourse) Question and answer Giving instructions and directions Paraphrasing Interview Role-play Discussion and conversation Games Oral presentation Picture-cued storytelling Retelling a story, news event Translation (of extended prose) Others (please specify): 1 1 1 1 1 1 1 1 1 1 1 72 Elicit very well 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 5 To what extent you think these types of speaking tasks can elicit samples of students’ ability to describe and make comparisons (with some clues) about learnt topics? Elicit very well Cannot elicit SPEAKING TASK Repetition Read-aloud Directed response Sentence/Dialogue completion task Picture-cued Translation (of limited stretches of discourse) Question and answer Giving instructions and directions Paraphrasing Interview Role-play Discussion and conversation Games Oral presentation Picture-cued storytelling Retelling a story, news event Translation (of extended prose) Others (please specify): 1 1 1 2 2 2 3 3 3 4 4 4 5 5 5 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 How likely you think these speaking tasks will be used by 8th graders in real-life English communication? SPEAKING TASK Very unlikely Repetition Read-aloud Directed response Sentence/Dialogue completion task Picture-cued Translation (of limited stretches of discourse) Question and answer Giving instructions and directions Paraphrasing Interview Role-play Discussion and conversation Games 1 1 1 1 1 1 73 Very likely 2 2 2 2 2 2 3 3 3 3 3 3 4 4 4 4 4 4 5 5 5 5 5 5 Oral presentation Picture-cued storytelling Retelling a story, news event Translation (of extended prose) Others (please specify): 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 5 5 5 5 To what extent are the majority of students interested in these types of speaking tasks when they are conducted in your speaking lessons? Not interested at all SPEAKING TASK Repetition Read-aloud Directed response Sentence/Dialogue completion task Picture-cued Translation (of limited stretches of discourse) Question and answer Giving instructions and directions Paraphrasing Interview Role-play Discussion and conversation Games Oral presentation Picture-cued storytelling Retelling a story, news event Translation (of extended prose) Others (please specify): 1 1 1 2 2 2 3 3 3 4 4 4 5 5 5 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5  THANK YOU  74 Very interested APPENDIX QUESTIONNAIRE FOR TESTING EXPERTS Hello! My name is Van Anh Nguyen, a senior student in FELTE, ULIS-VNU Thank you for participating in the questionnaire for my Graduation Paper named “The development and validation of an English speaking achievement test for 8th graders in a secondary school in Hanoi” The aim of this questionnaire is to gather testing experts’ opinions on the content validity of the speaking test utilized in the study Attached to this survey are the test specifications and the 8th grade book maps for your reference Your personal information and answers will be kept in strict confidentiality and will be serving research purposes only PERSONAL INFORMATION Years of experience: Email address: Phone number: I TASK – SOCIAL INTERACTIONS What is the difficulty of Task according to CEFR levels? (Please circle the option that matches your opinion) A A1 B A2 C B1 – C2 From question to 7, please tick the box that matches your opinion towards each statement Totally Disagree disagree   The task topic is relevant to the course content   The task can elicit evidence of students’ ability to demonstrate English sounds, stress patterns, rhythmic structure, and intonation contours in basic compound sentences   The task can elicit evidence of students’ ability to participate in short and simple exchanges of information about learnt topics   The instruction of the task is explicit to the students   The amount of interlocutor – candidate interaction is suitable for the task   The duration (1-2 minutes) is appropriate for conducting the task 75  Totally agree            Agree II TASK – GIVING DIRECTIONS What is the difficulty of Task according to CEFR levels? (Please circle the option that matches your opinion) A A1 B A2 C B1 – C2 From question to 9, please tick the box that matches your opinion towards each statement Totally disagree Disagree Agree Totally agree                             The task topic is relevant to the course content The task can elicit evidence of students’ ability to demonstrate English sounds, stress patterns, rhythmic structure, and intonation contours in basic compound sentences The task can elicit evidence of students’ ability to give simple directions in daily interactions The number of questions (1 question/candidate) is sufficient to meet the task objectives The instruction of the task is explicit to the students The picture prompt (map) is explicit to the students The amount of examiner – examinee interaction is suitable for the task The duration (1-2 minutes) is appropriate for conducting the task III TASK – EXCHANGING INFORMATION What is the difficulty of Task according to CEFR levels? (Please circle the option that matches your opinion) A A1 B A2 C B1 – C2 76 From question to 9, please tick the box that matches your opinion towards each statement Totally Totally Disagree Agree disagree agree The task topic is relevant to the course content The task can elicit evidence of students’ ability to demonstrate English sounds, stress patterns, rhythmic structure, and intonation contours in basic compound sentences The task can elicit evidence of students’ ability to participate in short and simple exchanges of information about learnt topics The task can elicit evidence of students’ ability to describe and make comparisons (with some clues) about learnt topics The instruction of the task is explicit to the students The cue card is explicit to the students The amount of examinee – examinee interaction is suitable for the task The duration (5-6 minutes) is appropriate for conducting the task                                 IV THE TEST IN GENERAL From question to 3, please tick the box that matches your opinion towards each statement Totally Disagree disagree   The duration of the test (7-10 th minutes) is appropriate to graders   The language of instruction th (English) is appropriate to graders   The designed tasks are sufficient to test the targeted objectives 77  Totally agree      Agree What are your recommendations for the test regarding its duration, language of instruction and number of tasks? V THE RATING SCALE To what extent you think the criteria in the rating scale are relevant to the test objectives? (Please circle the number that matches your opinion) Not relevant Need some revisions Relevant but need some minor revisions Very relevant What are your recommendations to make the rating scale criteria more relevant to the test objectives? To what extent you think the criteria in the rating scale are representative of the test objectives? (Please circle the number that matches your opinion) Not representative Need some revisions Representative but need some minor revisions Very representative 4 What you think are needed to make the rating scale criteria more representative of the test objectives? 78 To what extent you think the performance levels in the rating scale are relevant to the test objectives? (Please circle the number that matches your opinion) Not relevant Need some revisions Relevant but need some minor revisions Very relevant What are your recommendations to make the rating scale’s performance levels more relevant to the test objectives? To what extent you think the performance levels in the rating scale are representative to the test objectives? (Please circle the number that matches your opinion) Not representative Need some revisions Representative but need some minor revisions Very representative What are your recommendations to make the rating scale’s performance levels more representative to the test objectives? To what extent you think the language in the rating scale is explicit to its users? (Please circle the number that matches your opinion) Not explicit Need some revisions Explicit but need some minor revisions Very explicit 10 What are your recommendations to make the rating scale language more explicit to its user?  THANK YOU  79 APPENDIX OBSERVATION SCHEME FOR TEXTBOOK Name of instructional material: Observer: UNIT SPEAKING TASK PAGE OBJECTIVE(S)  demonstrate English sounds, stress patterns, rhythmic structure, and intonation contours  give simple directions in daily interactions  participate in short and simple information exchange about learnt topics  describe and make comparisons (with some clues) about learnt topics  demonstrate English sounds, stress patterns, rhythmic structure, and intonation contours  give simple directions in daily interactions  participate in short and simple information exchange about learnt topics  describe and make comparisons (with some clues) about learnt topics  demonstrate English sounds, stress patterns, rhythmic structure, and intonation contours  give simple directions in daily interactions  participate in short and simple information exchange about learnt topics  describe and make comparisons (with some clues) about learnt topics 80 APPENDIX OBSERVATION SCHEME FOR ENGLISH LESSON Lesson: Class: Teacher: Observer: Date: Time: No SPEAKING TASK PROCEDURE OBJECTIVE(S)  demonstrate English sounds, stress patterns, rhythmic structure, and intonation contours  give simple directions in daily interactions  participate in short and simple information exchange about learnt topics  describe and make comparisons (with some clues) about learnt topics  demonstrate English sounds, stress patterns, rhythmic structure, and intonation contours  give simple directions in daily interactions  participate in short and simple information exchange about learnt topics  describe and make comparisons (with some clues) about learnt topics 81 APPENDIX BOOK MAP OF ENGLISH TEXTBOOK Unit 1: Leisure activities Unit 2: Life in the countryside Unit 3: Peoples of Vietnam READING - Reading for general and specific information about the possible effects of spending too much time on the computer - Reading for specific information about an unusual lifestyle in the countryside: Mongolian nomadic life - Reading for specific information about an ethnic group REVIEW - Reading for specific Unit 4: Our customs and information about family customs and traditions traditions Unit 5: Festivals in Viet Nam - Reading for specific information about two festivals in Viet Nam LISTENING - Listening for specific information about ways of spending time with friends SPEAKING - Talking about “good” and “bad” sides of leisure activities WRITING - Writing to discuss an opinion about leisure activities LANGUAGE FOCUS - Verbs of liking + gerunds - Verbs of liking + to-infinitives - Clusters: /br/ and /pr/ - Listening for specific information about changes in the countryside - Talking about what you like or dislike about life in the countryside - Writing about changes in the countryside - Comparative forms of adjectives: review - Comparative forms of adverbs - Clusters: /bl/ and /cl/ - Listening for specific information about a traditional speciality - Talking about life of an ethnic group - Writing about how to cook a traditional dish - Questions: review - Articles (some uses) - Clusters: /sk/, /sp/ and /st/ - Listening for specific information about a traditional dance of an ethnic group in Viet Nam - Listening for general and specific information about a - Talking about family customs and traditions - Writing about a traditional dance - Should and shouldn’t: review - have to - Clusters: /spr/, /str/ - Talking about the reasons for going to a festival and the things - Writing about a festival - Simple sentences and compound sentences: review 82 traditional festival Unit 6: Folk tales REVIEW Unit 7: Pollution Unit 8: English speaking countries Unit 9: Natural disasters REVIEW Unit 10: Communication Unit 11: - Reading for specific information about a fairy tale - Listening for specific information about a fairy tale - Reading for general and specific information about water pollution - Listening for specific information about thermal pollution - Reading for specific information about an English speaking country to take when attending a festival - Talking about a legend folk tale/fairy tale/fable - Talking about causes and effects of water pollution as well as solutions to this problem - Narrating a legend/folk tale/fairy tale/fable - Past simple: review - Past continuous - Intonation in exclamatory sentences - Writing about the causes and effects of a pollution type - Describing a schedule for a visit or a tour - Conditional sentences type 1: review - Conditional sentence type - Pronunciation: stress in words ending in –ic and –al - Listening for specific information about a day trip to a town in an English speaking country - Reading for specific - Listening for specific information about how information about a to prepare for a natural natural disaster disaster - Talking about an English speaking country - Talking about natural - Writing a disasters and ways to news report prepare for them on a natural disaster - Passive voice: review - Past perfect - Pronunciation: stress in words ending in –logy and -graphy - Reading for specific information about a future way of communication - Reading for specific - Listening for specific information about netiquette - Talking about ways of communication now and in the future - Listening for specific - Expressing - Future continuous: review - Verbs + to-infinitive - Pronunciation: stress in words ending in –ity and –itive - Future tenses: review 83 - Present tenses: review - Present simple for future - Pronunciation: stress in words ending in –ese and –ee Science and technology information about future role of science and technology Unit 12: Life on other planets - Reading for specific information about life on another planet information about the benefits and problems science and technology may bring - Listening for specific information about people on another planet agreement and disagreement about the roles of science and technology - Talking about life on another planet - Reported speech (statements_ - Pronunciation: stress in words starting with un- and im- May and might: review - Reported speech (questions) - Pronunciation: stress in words ending in –ful and -less REVIEW Reference: Ministry of Education and Training (2017a) Tiếng Anh – Sách học sinh – Tập Một Can Tho: Vietnam Education Publishing House Ministry of Education and Training (2017b) Tiếng Anh – Sách học sinh – Tập Hai Can Tho: Vietnam Education Publishing House 84 ...ĐẠI HỌC QUỐC GIA HÀ NỘI TRƯỜNG ĐẠI HỌC NGOẠI NGỮ KHOA SƯ PHẠM TIẾNG ANH KHÓA LUẬN TỐT NGHIỆP XÂY DỰNG VÀ XÁC TRỊ BÀI KIỂM TRA HỌC KÌ KĨ NĂNG NĨI MƠN TIẾNG ANH CHO HỌC SINH LỚP TẠI MỘT TRƯỜNG THCS. .. MỘT TRƯỜNG THCS TẠI HÀ NỘI Giáo viên hướng dẫn: Dương Thu Mai, PhD Sinh viên: Nguyễn Vân Anh Khóa: QH2014.F1.E1 HÀ NỘI – 20 18 ACCEPTANCE PAGE I hereby state that I: Nguyễn Vân Anh, QH2014.F1.E1,... Cohesion Strategic capacity Achievement strategies  Overgeneralization  Paraphrase  Word coinage  Restructuring  Cooperative strategies  Code switching  Non-linguistic strategies Avoidance strategies