Evaluating the validity of the final achievement test for second year non major students at electronic electrical engineering department, namdinh university of technology education

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES FACULTY OF POST – GRADUATE STUDIES SUBMITTED BY: TRẦN THỊ THU HƯƠNG A thesis submitted in partial fulfillment of the requirements for the degree of Master of Arts EVALUATING THE VALIDITY OF THE FINAL ACHIEVEMENT TEST FOR SECOND – YEAR NON – MAJOR STUDENTS AT ELECTRONIC – ELECTRICAL ENGINEERING DEPARTMENT, NAM DINH UNIVERSITY OF TECHNOLOGY EDUCATION (Đánh giá độ giá trị kiểm tra cuối kỳ cho sinh viên không chuyên tiếng Anh năm thứ hai khoa Điện – Điện tử, Trường Đại học Sư phạm Kỹ thuật Nam Định) M.A MINOR THESIS Field: Language Teaching Methodology Code: 60 14 10 HANOI, 2011 VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES FACULTY OF POST – GRADUATE STUDIES SUBMITTED BY: TRẦN THỊ THU HƯƠNG A thesis submitted in partial fulfillment of the requirements for the degree of Master of Arts EVALUATING THE VALIDITY OF THE FINAL ACHIEVEMENT TEST FOR SECOND – YEAR NON – MAJOR STUDENTS AT ELECTRONIC – ELECTRICAL ENGINEERING DEPARTMENT, NAM DINH UNIVERSITY OF TECHNOLOGY EDUCATION (Đánh giá độ giá trị kiểm tra cuối kỳ cho sinh viên không chuyên tiếng Anh năm thứ hai khoa Điện – Điện tử, Trường Đại học Sư phạm Kỹ thuật Nam Định) M.A MINOR THESIS Field: Language Teaching Methodology Code: 60 14 10 Supervisor: Phạm Lan Anh, M.A HANOI, 2011 vi TABLE OF CONTENTS DECLARATION i ACKNOWLEDGEMENTS ii ABSTRACT iii LIST OF FIGURES, TABLES AND CHARTS iv LIST OF ABBREVIATIONS v TABLE OF CONTENTS .vi CHAPTER 1: INTRODUCTION 1.1 Rationale 1.2 Scope of the study 1.3 Aims of the study 1.4 Methods of the study 1.5 Research questions 1.6 Design of the study CHAPTER 2: LITERATURE REVIEW 2.1 Relationship between teaching, learning and assessment 2.2 Purposes of formative and summative assessments 2.3 Achievement tests and their characteristics 2.3.1 Achievements tests 2.3.2 Characteristics of a good EGP test 11 2.3.3 Characteristics of a good ESP test 14 2.4 Face validity 15 2.4.1 Definition 15 2.4.2 Relationship between reliability and validity 16 2.4.3 Reasons for choosing face validity 17 2.5 Some measures to increase face validity 18 vii CHAPTER 3: THE STUDY 20 3.1 English learning and teaching at Nam Dinh University of Technology Education 20 3.1.1 Students’ backgrounds 20 3.1.2 The English teaching staff 20 3.1.3 Objectives of the English course 21 3.1.4 Checklist of the course book 22 3.1.5 Objectives of the final test 23 3.1.6 Difficulty level and discrimination of the final test 24 3.2 English testing at Nam Dinh University of Technology Education 24 3.2.1 Testing situation 24 3.2.2 The current final achievement test 25 3.3 Research methods 26 3.3.1 Survey questionnaire 26 3.3.2 Interview and informal discussion 26 3.4 Data analysis of survey questionnaires and interviews 26 3.4.1 Data analysis of the administration of the test 27 3.4.1.1 Data analysis of the format of the test 27 3.4.1.2 Data analysis of the logistics of the test 28 3.4.2 Data analysis of face validity of the test 29 3.4.2.1 Data analysis of general opinion about the test 30 3.4.2.2 Data analysis of reading comprehension task 31 3.4.2.3 Data analysis of grammar knowledge task 33 3.4.2.4 Data analysis of translation task 34 3.5 Discussion and findings 36 3.5.1 Similarities in teachers and students’ perception 36 3.5.1.1 Test administration 36 viii 3.5.1.2 Face validity 36 3.5.1.2.1 General opinion about the test 36 3.5.1.2.2 Reading comprehension task 36 3.5.1.2.3 Grammar task 37 3.5.1.2.4 Translation task 37 3.5.2 Differences in teachers and students’ perception 37 3.5.2.1 Test administration 37 3.5.2.2 Face validity 37 3.5.2.2.1 Grammar task 37 3.5.2.2.2 Translation task 37 3.6 Suggestions to improve the final achievement test 38 CHAPTER 4: CONCLUSION 41 REFERENCES 42 APPENDICES I APPENDIX I APPENDIX V APPENDIX IX APPENDIX XIII iv LIST OF FIGURES, TABLES AND CHARTS Figures Figure 1: Three Considerations for Test Choice Figure 2: The Scope of Impact of Language Tests Figure 3: Relationship between reliability and validity Tables Table 1: ESP syllabus content allocation Table 2: Specification of test 12 Table 3: Teachers and students’ comments on the format of the test Table 4: Teachers and students’ comments on the administration of the test Table 5: Teachers and students’ comment on the whole test Table 6: Teachers and students’ comment on students’ reading comprehension ability, theme and instruction of the reading comprehension task Table 7: Teachers and students’ comment on the grammar task Table 8: Teachers and students’ comment on the translation task Charts Chart 1: Percentage of teachers and students’ comment on what language ability the test mainly intend to measure Chart 2: Percentage of students and teachers’ opinions on what test items can measure their true ability Chart 3: Percentage of the results which students get in the test Chart 4: Percentage of what test tasks students cannot Chart 5: Percentage of teachers and students’ comment on the length of the reading text Chart 6: Students’ comment on whether or not the reading text is difficult Chart 7: Teachers’ comment on which types of reading skills are expressed in the reading comprehension task Chart 8: Teachers and students’ comment on which students’ ability this translation task requires v LIST OF ABBREVIATIONS NUTE: Nam Dinh University of Technology Education EGP: English for General Purposes ESP: English for Specific Purposes CHAPTER 1: INTRODUCTION 1.1 Rationale Learning English in Vietnam today is quite popular and its popularity is increasing day by day This is due to the fact that Vietnam has recently adopted an open-door policy which encourages broadening and improving its relationship and cooperation with other countries in many aspects of life such as diplomatic, economic, cultural, scientific and technological areas English is not only a means but also a key to gaining access to latest scientific and technological achievements for a developing country like Vietnam, where modern science and technology is badly needed Recognizing the necessity of this global language, most of the schools, colleges and universities in Vietnam consider English as the main, compulsory subjects that students must learn In the English language learning and teaching process, evaluating a test is significant Test is a part of teaching and learning process because it also provides feedback about the achievement of teaching and learning objectives for those who are involved in the education system Moreover, the use and necessity of knowing a person’s language ability is through a language test In education, especially at the Faculty of Foreign Languages at Nam Dinh University of Technology Education (NUTE), testing the students’ achievement toward teaching objectives is needed Without an achievement test, it is difficult to see how rational educational decisions can be made NUTE is a technological university and students’ learning English ability is really low Evaluating a test is also one of the methods to improve students’ learning process and results Although there were many theses mentioned this problem but at NUTE it is still new and very necessary because how to evaluate the test after each semester still receives little attention and up to now the process of test analysis after each examination hasn’t been fully investigated Consequently, students’ results are getting worse and worse As a teacher myself, I see that we, teachers at NUTE, just stop at experienced level of test making procedure, test administration, test marking procedure during and after examination We not care testing evaluation from other teachers and students, so the results of test are not still improved Moreover, test’s validity can be seen as an attempt for improving the test quality Being a measure of students’ achievement toward learning objectives, final examinations must be valid Validity is one of the characteristics of a qualified test Therefore, “Evaluating the validity of the final achievement test for second – year non – major students at Electronic – Electrical Engineering Department, Nam Dinh University of Technology Education” is chosen with the hope that the study will be helpful with the author, the teachers, the test-takers and everyone who is concerned with language testing in general and validity of an achievement test in particular Due to limit time in collecting students’ scores, this study is different from the previous study The author only focuses on the face validity of this test The author hopes that the result of the study can then be applied to improve the current test and to create a new really reliable item bank It is also intended to encourage both teachers and learners in their teaching and learning 1.2 Scope of the study The scope of this thesis is limited to a research on teachers’ and test-takers’ evaluation of the existing achievement test in terms of its face validity for the second-year non-English major students at Electronic – Electrical Engineering Department, NUTE due to the limitations in time, ability and availability of data Moreover, it is impossible for the author to cover all used final achievement tests as well as design a sample achievement test for second-year students Instead, only a test specification for test 12 in semester is presented 1.3 Aims of the study Following the scope of the research above, the aims of this research are: To indentify the English teachers and students’ evaluation of the final existing achievement test (test 12) at NUTE in terms of face validity To provide suggestions for test designers 1.4 Methods of the study In order to achieve the above aims, the study has been carried out as follows: First, the author goes to library to read theory about assessment and testing, achievement test with characteristics of a good achievement test and test validity with a special focus on face validity and some measures to increase it From her critical reading, many reference materials have been gathered, analyzed, and synthesized to draw out a rd theoretical basis to evaluate the current test being used for the semester students in terms of its face validity Then, qualitative methodologies involving data collected through survey questionnaires and interviews were employed from both teachers and students at NUTE 1.5 Research questions This study is implemented to find answers to the following research questions: What are the teachers’ and test takers’ (students’) perceptions of the final rd semester English achievement test at NUTE in terms of its face validity? rd What are suggestions to improve face validity of the final semester English achievement test at NUTE? 1.6 Design of the study The thesis is divided into four major chapters: Chapter 1: Introduction presents basic information such as: the rationale, the scope, the aims, the method, the research questions and the design of the study Chapter 2: Literature review reviews theoretical backgrounds on evaluating a test, which includes relationship between teaching, learning and assessment, purposes of formative and summative assessments, achievement tests, characteristics of good EGP and ESP tests, face validity and some measures to increase face validity Chapter 3: The study is the main part of the thesis showing the context of the study and the detailed result obtained from collected tests and findings in response to the research questions Then, the author gives some solutions to improve the final achievement test Chapter 4: Conclusion offers conclusions and proposes some suggestions for further research on the topic IX Đề số 12 I Read the text and decide if these statements are true (T) of false (F) (2.5 pts) In the first category are substances which provide an easy path for an electric current All metals are conductors However, some metals not conduct well Maganin, for example, is a poor conductor Copper is a good conductor, therefore it is widely used for cables A non-metal which conducts well is carbon Salt water is an example of a liquid conductor A material which does not easily release electrons is called an insulator Rubber, nylon, porcelain and air are all insulators There are no perfect insulators All insulators will allow some flow of electrons However, this can usually be ignored because the flow they permit is so small Semiconductors are midway between conductors and insulators Under certain conditions, they allow a current to flow easily but under others, they behave as insulators Germanium and silicon are semiconductors Mixtures of certain metallic oxides also act as semiconductors These are known as thermistors The resistance of thermistors falls rapidly as their temperature rises They are therefore used in temperature-sensing devices …… All good conductors are non – metals …… Copper does not allow an electric current to flow easily …… Rubber provides an uneasy path for an electric current …… Porcelain is not a perfect insulator …… The resistance of a thermistor is higher at low temperature than at high temperature X II Give the correct passive forms of the verbs in brackets (2.5 pts) A pn junction is formed if two crystals of a semiconductor material, one of p-type and one of n-type (join) together Radio frequency waves (use) to carry audio frequency waves over long distances through the air For a long time, hi-fi recording (produce) on vinyl gramophone records Electrolysis (can use) to break down water into hydrogen and oxygen The electric relay works on the principle that if the magnetic field is cut by another conductor, an electromotive force (will enduce) in that conductor III Translate into Vietnamese (3.0 pts) A typical radio tuner circuit consists of an inductor and capacitor connected in parallel The side of the aerial inductance coil can be kept small by winding it on a ferrite rod core The RF waves fed to the tuner cause the circuit to oscillate The impedance of the circuit is smaller and the oscillator is greatest at a particular frequency known as the resonant frequency This frequency is determined by the values of the inductance and the capacitance By using a variable capacitor, the circuit can be tuned to the required radio frequency and the selected RF wave passed on to the RF amplifier IV Translate into English (2.0 pts) Động điện máy biến đổi điện sang thành Khi dòng điện chạy qua dây dẫn, làm nóng dây dẫn Một máy dao động không cho biết độ lớn tín hiệu mà cịn cho biết thay đổi tín hiệu theo thời gian Đĩa ghi âm sử dụng hệ thống ghi âm mô phỏng, hệ thống lưu mẫu hình việc tạo đường rãnh liên tục đĩa nhựa vinyl Nam Định, ngày 31 tháng 12 năm 2010 TRƯỞNG BỘ MÔN XI TRƯỜNG ĐẠI HỌC SPKT NAM ĐỊNH ĐÁP ÁN ĐỀ THI KẾT THÚC HỌC PHẦN KHOA NGOẠI NGỮ TÊN HỌC PHẦN: Tiếng Anh chuyên ngành TRÌNH ĐỘ ĐÀO TẠO: Đại học NGÀNH: Các ngành điện Thời gian: 60 phút Đề số 12 Questions I Read the text and F F T T T II Give the correct passive form are joined are used have been produced can be used will be enduced III Translate into Vietnamese Mạch điện phím điều tụ điện mắc song song Dây ăng ten tự cảm đượ lõi pherit 0.5 Sóng tần số vơ tuyến nạp vào phím điều chỉnh làm cho mạch dao động Ở tần số cụ thể, hay gọi mạch nhỏ dao động Các giá trị tự cảm điện dung q Bằng cách sử dụng tụ xoay, số vô tuyến theo yêu cầu sóng tiếp đến thiết bị khuyếch đại són XII IV Translate into E An electric moto mechanical energ When a current f An oscilloscope signal varies with Records use an a cutting a continu XIII APPENDIX 3.4 Data analysis of survey questionnaires and interviews 3.4.1 Data analysis of the administration of the test 3.4.1.1 Data analysis of the format of the test Table 3: Teachers and students’ comments on the format of the test Font Times New Roman and size 12 of the test Copies of the test … The quantity of questions of the test … Can you give more suggestions for the improvement of test format? 3.4.1.2 Data analysis of the logistics of the test Table 4: Teachers and students’ comments on the administration of the test Options Questions Time allowance for this test … Are rooms for testing big enough? The quantity of the test paper … Do teachers have measures to prevent cheating? The atmosphere of XIV the rooms for testing … 10 Can you give more suggestions for the improvement of logistics of the test? 3.4.2 Data analysis of face validity of the test 3.4.2.1 Data analysis of general opinion about the test 80% 70% 60% 50% 40% 30% 20% 10% 0% Chart 1: Percentage of teachers and students’ comment on what language ability the test mainly intend to measure translation into Vietnamese translation into English grammar reading 0% Chart 2: Percentage of students and teachers’ opinions on what test items can measure their true ability XV s tudent 35% mark 1-6 mark 7-10 65% Chart 3: Percentage of the results which students get in the test 14% s tudent 16% reading grammar 16% trans lation into E nglis h trans lation into Vietnames e 54% Chart 4: Percentage of what test tasks students cannot Table 5: Teachers and students’ comment on the whole test 14 Should you adjust parts of the test? 15 students? Is the test suitable with XVI 3.4.2.2 Data analysis of reading comprehension task Table 6: Teachers and students’ comment on students’ reading comprehension ability, theme and instruction of the reading comprehension task Options Questions 16 Does the reading task really test students’ reading comprehension? 17 Does theme in the reading text have in the syllabus? 18 The instruction of the reading text is … 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Chart 5: Percentage of teachers and students’ comment on the length of the reading text XVII s tudent 38% yes no 62% Chart 6: Students’ comment on whether or not the reading text is difficult s kimming Chart 7: Teachers’ comment on which types of reading skills are expressed in the reading comprehension task XVIII 3.4.2.3 Data analysis of grammar knowledge task Table 7: Teachers and students’ comment on the grammar task Options Questions 22 The grammar task given in the test is … 23 The instruction of the grammar task is … 24 What you think of the difficulty level of the grammar task? 25 What you think of the content of the grammar task? XIX 3.4.2.4 Data analysis of translation task 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Chart 8: Teachers and students’ comment on which students’ ability this translation task requires XX Table 8: Teachers and students’ comment on the translation task Options Questions 27 The instruction of the translation tasks is … 28 Is the content of the translation tasks appropriate to the students’ ability? 29 Are there too many new words in the translation tasks? 30 Are structures in the translation tasks simple to students? ... qualified test Therefore, ? ?Evaluating the validity of the final achievement test for second – year non – major students at Electronic – Electrical Engineering Department, Nam Dinh University of Technology. .. requirements for the degree of Master of Arts EVALUATING THE VALIDITY OF THE FINAL ACHIEVEMENT TEST FOR SECOND – YEAR NON – MAJOR STUDENTS AT ELECTRONIC – ELECTRICAL ENGINEERING DEPARTMENT, NAM DINH UNIVERSITY. .. questions for students and teachers), and translation task (5 questions for students and teachers) 3.4.1 Data analysis of the administration of the test 3.4.1.1 Data analysis of the format of the test