Proposed improvement of informatics reading comprehension tests for hanoi university of technology

HANOI UNIVERSITY OF FOREIGN STUDIES r POST-GRADUATE STUDIES DEPARTMENT it PROPOSED IMPROVEMENT OF INFORMATICS READING COMPREHENSION TESTS FOR HANOI UNIVERSITY OF TECHNOLOGY f -v imsỵÂM T H Ư N R TIN I H 1/ V I Ê N *== z— - ^ ===~==J) BY HO XUAN NGOC THIS DISSERTATION IS PRESENTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS (TESOL) LL, Novem ber 1999 S U B M IS S IO N O F D IS S E R T A T IO N Title of Dissertation : p r o p o s e d im p r o v e m e n t o f in f o r m a t ic s r e a d in g COMPREHENSION TESTS FOR HANOI UNIVERSITY OFTECHNOLOGY Supervisor : Le Huy Truong, M.A Name of Candidate: Ho Xuan Ngoc Home address : B15, Yen Lang, Dong Da, Hanoi Tel : 531884 Date of start July 1999 Date of Submission November 1999 HANOI UNIVERSITY OF FOREIGN STUDIES POST-G RA DU A TE STUDIES DEPARTM ENT PROPOSED IMPROVEMENT OF INFO RM ATICS READING COMPREHENSION TESTS FOR HANOI UNIVERSITY OF TECHNOLOGY by HO XUAiN NGOC THIS DISSERTATION IS PRESENTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS (TESOL) N ovem ber I 1999 Certifícate of Originality of Field Study Report I certify my authorship of the Field Study Report submitted today entitled PROPOSED IMPROVEMENT OF INFORMATICS READING COMPREHENSION TESTS FOR HANOI UNIVERSITY OF TECHNOLOGY in terms of the Statement of Requirements for Theses and Field Study Reports in Masters' Programmes issued by the Higher Degree Committee Ho Xuan Ngoc November, II 1999 ABSTRACT That this field study report has been carried out originates from the fact that urgent remedies are needed to improve ESP reading comprehension tests at Hanoi University of Technology This field study report consists of five chapters as follows : Chapter one gives an overview of ESP testing at Hanoi University of Technology The aims of this field study report are also given in this chapter Chapter two covers questions such as the role of testing, types of language tests, and main characteristics of a good test Chapter three considers the problems concerning ESP, for example, a definition of ESP, description of ESP materials for students of informatics and methods used to teach those materials Chapter four takes into consideration such questions as the characteristics of reading tests, identification of test specifications, techniques for designing reading comprehension tests, problems of existing ESP reading tests at HNUT together with comments, and suggestions for improvement Chapter five reconfirms the aim of this field study report-to improve ESP reading tests at HNUT Ill ACKNOWLEDGEMENTS I am particularly indebted to my supervisor Mr Le Huy Truong, M.A for his valuable instructions, comments, criticism, correction, and for his kind encouragement during the development of this field study report Without his help this field study report would have been impossible I would also like to acknowledge a debt to Mr Tran Quang Huy, M.A, Dr Nguyen Ngoc Hung, and Dr Pham Kim Ninh I am grateful to all my lecturers, teachers and the staff of the Post­ graduate Studies Department at HUFS for their encouragement and assistance My gratitude also goes to my wife, Mrs Tran Thi Kim Lien, who is always with me in the sweet or bitter minutes of my life, whose support for writing this dissertation cannot be expressed in words, making a great contribution to my work I would also like to convey my thanks to my classmates for their help Finally, I would like to thank my collègues in the Faculty of Foreign Studies at HNUT for their support and encouragement IV TABLE OF CONTENTS Page acknow ledgem ents ABSTRACT TABLE OF CONTENTS CHAPTER INSTRUCTION CHAPTER THEORETICAL REVIEW OF TESTING 2.1 W hat is testing? 2.2 W hat is a test ? 2.3 The role of te s tin g 2.3.1 Testing and te a c h in g 2.3.2 Testing and te a c h e rs 2.3.3 Testing and s tu d e n ts 2.4 T est classific atio n 2.4.1 Diagnostic t e s t s 2.4.2 Placem ent t e s t s 2.4.3 Progress t e s t s 2.4.4 Achievem ent te s t s 10 2.4.5 Proficiency t e s ts 11 2.4.6 Aptitude te s t s 11 2.5 T est ch aracteristics 11 2.5.1 V alid ity 12 2.5.1.1 Face v a lid ity 12 2.5.1.2 Content v a lid ity 13 2.5.2 R elia b ility 13 2.5.2.1 Test - r e te s t 14 2.5.2.2 Internal c o n siste n c y 15 2.5.2.3 Split - h a l f .16 2.5.3 P racticality 17 2.5.4 D iscrim in atio n 18 V 2.5.4.1 Item difficulty 18 2.5.4.2 Item discrim ination 19 CHAPTER ENGLISH FOR SPECIFIC PURPOSES 21 3.1 Some understanding of ESP in general 21 3.2 ESP teaching to students of informatics at HNUT 23 3.2.1 Teaching m aterials 23 3.2.2 Teaching m ethods .28 CHAPTER HOW TO IMPROVE ESP READING COMPREHENSION TESTS FOR STUDENTS OF INFORMATICS AT HNUT 32 4.1 Characteristics of reading te sts 32 4.2 Identification of test specifications 33 4.3 Techniques used for designing reading comprehension tests 36 4.3.1 Multiple - choice te s t 36 4.3.2 True/False te s t 40 4.3.3 Rearrangement ite m s 43 4.3.4 Matching te s t .45 4.3.5 Completion test 47 4.3.6 Cloze test 48 4.4 Problems of existing reading tests at HNUT 50 4.5 Proposed improvement of existing tests 51 4.5.1 Procedures for test construction 51 4.5.2 Proposed improvements of the HNUT existing te s ts .52 CHAPTER CONCLUSION 59 APPENDIX A 61 APPENDIX B 62 BIBLIOGRAPHY 66 VI CHAPTER ONE INTRODUCTION At Hanoi University of Technology (HNUT) students are studying English in two stages of semesters (each semester is of 15 weeks) During the first stage (the first four semesters) the students are provided with 300 lesson periods o f general English (each period is of 45 minutes) They are expected to acquire pre-intermediate level through learning the first two books of the Headway series (Headway Elementary and Headway PreIntermediate) and some reading materials designed by the teachers of English at the Faculty of Foreign Studies During the second stage students are provided with 90 lesson periods of ESP At this stage ESP is taught to the students of different faculties : chemistry, physics, mathematics, information technology, electronics and telecommunications etc The reality of ESP teaching and learning at Hanoi University of Technology has supported the fact that there cannot be a universal ESP for all faculties, but there should be at least one kind of ESP for each faculty, for example, English for Computing At the Faculty of Information Technology students are studying English in the third year They are required to the practice of the two macro-skills : reading and writing In the middle of each academic year the students have to take a test, the mid-term test and at the end of the academic year they have to take the achievement test These tests often include subtests in reading and subtests in writing Section A: Read the following passage and choose the most suitable answer by circling the letter a, b, c, or d (Time allowed : 20 minutes) An expert on automation said that in the future, computers would be developed which would be small enough to carry in the pocket Ordinary people would then be able to use them to obtain valuable information Computers could be plugged into a national network and be used like radios For instance, people going on holiday could be informed about weather conditions; car drivers could be given alternative routes when there are traffic jams It will also be possible to make tiny translating machines This will enable people who don't share a common language to talk to each other without any difficulty or to read foreign publications It is impossible to assess the importance o f a machine o f this sort, fo r many international misunderstandings are caused simply through our failure to understand each other Computers will also be used in hospitals By providing a machine with a patient's symptoms, a doctor will be able to diagnose the nature o f his illness Similarly, machines could be used to keep a check on a patient's health record and bring it up to date Doctors will therefore have immediate access to a great many facts which will help them in their work Book-keepers and accountants, too, could be relieved o f dull clerical work, fo r the dull, routine task o f compiling and checking lists o f figures could be done entirely by machines Computers are the most efficient servants man has ever had and there is no limit to way they can be used to improve our lives In the future people who go on holiday abroad will be able to a) Plan their routes without maps and enjoy good weather and route 53 b) Share a common language with anyone they meet c) Talk to a foreigner in his own language and avoid misunderstandings d) Read foreign newspapers even i f they not know foreign languages Why will computer be useful to doctor ? a) Because they will relieve them o f dull clerical work b) Because they will provide them with information which will help them in their work; c) Because much o f the tedious work they have to will be done entirely by machines; d) Because they are up to date For instance, people going on holiday could get weather a) announcements b) reports c) notices d) conditions The importance o f a machine o f this sort is a) without doubt b) out o f reach c) out o f bounds d) beyond measure Book-keepers and accountants, too, could be having to dull clerical work a) relieved at b) prevented from c) discredited with d) freed from 54 Section B : Read the following passage and decide whether the statements below it are true (T) or false (F) by putting T, or F in each box (Time allowed : 20 minutes) Mainframes are those computer systems found in computer installations processing immense amounts o f data These powerful computers make use o f very high-speed main memories into which data and programs to be dealt with are transferred fo r rapid access These powerful machines have a larger repertoire o f more complex instructions which can be executed more quickly These computers can be o f two types: digital or analog The digital computer or general-purpose computer as it is often known, makes up about 90 per cent o f the large computers now in use The digital computers is like a gigantic cash register in that it can calculations in steps, one after another at tremendous speed and with great accuracy The analog computer works something like a car speedometer, in that it continuously works out calculations Both o f these computer types - the digital and the analog - are made up o f electronic components that may require a large room to accommodate them At present, the digital computer is capable o f doing anything the analog once did Moreover, it is easier to program and cheaper to operate A new type o f scientific computer system called the hybrid computer has now been produced that combines the two types into one Really powerful computers continue to be bulky and require special provision fo r their housing This is because much more space is taken up by the input/output devices than by the electronic components that not make up the bulk o f the machine in a powerful installation The power 55 consumption o f these machines is also quite high, not to mention the price that runs into hundreds o f thousands o f dollars The future will bring great developments in the mechanical devices associated with computer systems T F O l A mainframe is the type o f computer that can sit on top o f a desk Mainframes are very powerful and can execute jobs very rapidly and easily Digital computers are used more than analog computers 0 The analog computer is fa r smaller than a digital computer and therefore occupies very little space The hybrid computer is a combination o f both the digital and the analog computer The analog computer does its calculations one step at a time The digital computer continuously works out calculations 0 0 S Mainframes are huge powerful machines whose peripheral equipment takes up a lot o f space 0 Mainframes are expensive to buy and to operate O 10 Mainframe technology has reached the end o f the road No further development is needed Section C: Fill each of the numbered spaces with a suitable word (Time allowed : 10 minutes) Almost every profession now uses computers (1) areused fo r all kinds o f (2) For example, scientists use them So (3) mathematicians and economists Business people use (4) to make 56 business decisions Doctors also (5) computers to help their patients Computers (6) work easier fo r writers, too And (7) students now use computers to help (8) their studies Finally, some people use (9) at home to help them make (10) and pay bills Everyone knows that (11) are useful And there are many (12) new computer programs But there are (13) reasons to like computers Some people (1 ) the way computers hum and sing (15) they are working It is a (16) sound, like the sounds o f toys (17) childhood Computers also have lights and (18) pictures With a computer anyone can (19) like an artist And computers even (20) to have personalities That may sound strange, but computers seem to have feelings Sometimes they seem happy, sometimes they seem angry It is easy to think they are like people Section D : Match the words in column A with their definitions in column B by putting the letter a, b, c, or / corresponding to the numbers in the boxes given below it (the first one has been done for you as an example) (Time allowed : 10 minutes) A B stylus a a semiconductor chip that form s the central processor o f a computer voice recognition logical record b an electronic device used to draw or write on the screen c an individual dot on a computer screen The computer controls the colour and brightness o f each pixel automaton d the technology that allows a computer to 57 interpret human speech This is a part o f artificial intelligence studies e a software system designed to handle multiple graphies requests fo r data access while at the same time maintaining the integrity o f the data f the collection o f data in a database relating to artificial one subject intelligent i the logical design o f a database conceptual schema j a machine capable o f operating independently, pixel such as a clothes drier k computer programmes that perform tasks which micro require intelligence when done by humans processor I a non-character based method o f displaying 10 database information on a screen, usually used fo r management displaying pictures The basic unit from which the display is built up is the pixel b 58 10 CHAPTER FIVE CONCLUSION In this Field Study Report I have presented my understanding of English language testing in general, and ESP testing in particular I have also discussed problems concerning ESP testing that HNUT teachers are now facing In my opinion, the most serious problems are : • Lack of a test bank for students of informatics, • Lack of urgent training in testing for HNUT English teachers, • Lack of the participation o f informatics experts in designing tests I have considered in chapter four a number of techniques for designing reading tests and have also offered some test samples which may be used for students of informatics at HNUT I have analysed some reading tests designed and administered by HNUT teachers and proposed some suggestions to improve them One more important thing which should be taken into account in this chapter is some practical measures to settle problems that are now facing ESP testing at HNUT My proposed practical measures are as follows : • Before designing any new tests, necessary remedies should be carried out to make the existing ones better • The designing of the new test should be based on basic principles for test design, and the cooperation between language teachers and the speciality teachers • Those who take part in designing the test should have good knowledge of language testing, especially ESP testing 59 • Like ESP teachers, ESP test designers should be required to understand the area of speciality with which they are concerned • Revision of new tests should be done after pilot application to ensure that the new tests will possess the characteristics of a good test My proposed solution in this field study report cannot escape from shortcomings, but I hope my suggestions will make a certain contribution to the improvement of ESP testing for Hanoi University of Technology 60 APPENDIX A MEAN “Mean” is calculated by the following formula m = — where: N m = Mean; I = total score of all testees N = number of testees let £ = 702; N = 26; then we have m = 26 = 24 “ Standard deviation” (s.d) is calculated as follows : s.d = d2 — N where s.d = Standard Deviation d = difference between one testee’s total scores - Mean = di" + + d3~ + dn~ N = number of testees Note: n = N let N = 26; d2 = 432; then we have s.d = 61 26 = 4.08 APPENDIX B The HNUT reading tests Section A Read the passage then choose the best answers to the questions that follow by circling a, b, c or d (10 points) Informatics (in Europe) or computer science (in the United States or related countries) has become a world-wide accepted science since the second h a lf or the last third o f the current century There is a study at university level There are congresses, scientific books and journals, scientific societies on national and international base But it is not easy to define what Informatics is The same problem can be found in Mathematics and some other sciences Now an attempt fo r definition is given: Informatics is the science o f information and information handling in nature, techniques and society Often the definition is more restricted to machine-based or computer-based and restricted to techniques with the sense o f engineering But we can see in the history o f human being the roots o f information handling is very old Different methods o f writing, storing, coding, transmitting o f information were found long before any computers exist Moreover, long before the existence o f mankind we can see in the living world o f plants and animals a genetic information handling Within the genes o f living beings, all features and most o f the behaviour are coded and can be varied from one generation to the next The science o f information studies these methods in the history and tries to use these fo r the construction o f devices or the developing o f methods What does this passage mainly discuss ? 62 a) Definition o f informatics b) History o f human beings c) Methods o f processing information d) New ideas fo r new devices' construction According to the passage, the world has accepted informatics as a science a) fo r the second time in this century c) both a and b b) fo r the third time in this century d) fo r about 30 to 50 years According to the passage, informatics is more often referred to as a) engineering c) something based on computer b) something restrictive d) restriction o f computer applications As used in the passage context, the word handling is similar in meaning to which o f the following ? a) processing c) organizing b) managing d) controlling According to the passage, which o f the follow ing is true ? a) All actions o f living beings are coded in their genes b) All characteristics o f living beings are coded in their genes c) Both a and b d) One generation o f living beings is varied from the next one Section B Draw a diagram showing the structure o f the paragraph below Point out its main idea, major and minor details (5 points) All computers, whether large or small, have the same basic capabilities They have circuits fo r performing arithmetic operations They all have a way o f communicating with the person(s) using them They also have circuits fo r making decisions 63 Section C Fill in each gap with the correct fo rm o f the word given (10 points) In 1983 a new type o f phonograph disc (reach) the marketplace, which showed (promise) o f eventually (take) over the recording industry It was called the compact disc and (make) use o f two technical systems : (digit) recording and laser beams All recording until the advent o f the compact disc was analog recording, which recorded the (vibrate o f sound waves on a track With analog recording, there was a bit o f slippage, which added (distort) Digital recording is a (complete) different process, one in which the distortion is so small that it can be said to be effectively (eliminate) Thus the fidelity o f the compact disc i s 10 (extreme) high Section D Find the appropriate terms fo r the definitions given (10 points) a semiconductor chip that form s the central processor o f a computer an electronic device used to draw or write on the screen an individual dot on a computer screen The computer controls the colour and brightness o f each pixel _ the technology that allows a computer to interpret human speech This is a part o f artificial intelligence studies a software system designed to handle multiple requests fo r data access while at the same time maintaining the integrity o f the data the collection o f data in a database relating to one 64 subject _the logical design o f a database _a machine capable o f operating independently, such as a clothes drier 9. computer programmes that perform tasks which require intelligence when done by humans 10 _ a non-character based method o f displaying information on a screen, usually used fo r displaying pictures The basic unit from which the display is built up is the pixel 65 BIBLIOGRAPHY - Alderson, J.C., 1981, Report o f the Discussion on Communicative Language Testing - Alderson, J.C., 1981, Issues in Language Testing, ELT Documents 111, London, the British Council - Alderson, J.C., 1986, Innovations in Language Testing, M portal (ed) - Alderson, J.C., 1995, Language Test Construction and Evaluation, Cambridge University Press - Bachman, L.F and Palmer, A.S 1996, Language Testing in Practice, Oxford - Baker, D., 1989, Language Testing: A Practical Review and Practical Guide, Oxford University Press - Brown, F.G., 1972, Measurement and Evaluation, E.F Peacok Publishers - Brown, H.D., 1980, Principles o f Language Learning and Teaching, New Jersey: Prentice-Hall - Bruck, C., 1976, Techniques and Problems o f Assessment, Hong Kong University - Caroll, J.B., 1980, Testing Communicative Performance, Oxford University Press - Cohen, A.D., 1980, Testing Language Ability in the Classroom, Newbury House - Davies, A., 1990, Principles o f Language Testing, Cambridge, Mass, Basil Blackwell - Harmer, J., 1996, The Practice o f English Language Teaching, Longman - Harris, D.P., 1983, Publication on Language Testing, Oxford University Press 66 - Harrison, A., 1983,/! Language Testing Handbook, London: Macmillan - Heaton, J.B., 1989, Writing English Language Tests, London, New York - Hughes, A., 1989, Testing for Language Teachers, Cambridge University Press - Hutchinson T., and Waters A.m 1993, English for Specific Purposes, Cambridge University Press - Ingram, E., 1968, Attainment and Diagnostic Testing, Oxford University Press - Jack C Richards, John Platt, Heidi Platt 1993, Longman dictionary of Language teaching and applied Linguistics - Kennedy C., and Bolitho R., 1991, English for Specific Purposes, Macmillan Press LTD - Lado, R., 1961, The Construction and Use o f Foreign Language Tests, Longman - Lee, Y., 1985, New Directions in Language Testing, Oxford: Pergamon - Le Huy Truong, 1988, Towards Improving ESP Testing in Vietnam, A Field Study Report for M.A Degree TESOL - Madsen, 1983, Techniques in Testing, Oxford University Press - Munby, L.J., 1978, Communicative Syllabus Design, Cambridge University Press - Murphy, M.J., 1969, Designing Multiple-Choice Items for Testing English Language, African University Press, Lagos - Oiler, J.W., 1979, Language Tests at School, Longman Group, London - Oiler, J.W., 1983, Cloze Tests o f Second Language Proficiency and What They Measure, A Journal of Applied Linguistics, pp 105-18 - Robinson P., 1991 ESP Today, Prentice Hall International (UK) Ltd - Valette, R.M., 1977, Modern Language Testing, Second Edition, Harcourt, Brace, Jovanovich 67 .. .HANOI UNIVERSITY OF FOREIGN STUDIES r POST-GRADUATE STUDIES DEPARTMENT it PROPOSED IMPROVEMENT OF INFORMATICS READING COMPREHENSION TESTS FOR HANOI UNIVERSITY OF TECHNOLOGY f -v... authorship of the Field Study Report submitted today entitled PROPOSED IMPROVEMENT OF INFORMATICS READING COMPREHENSION TESTS FOR HANOI UNIVERSITY OF TECHNOLOGY in terms of the Statement of Requirements... Date of Submission November 1999 HANOI UNIVERSITY OF FOREIGN STUDIES POST-G RA DU A TE STUDIES DEPARTM ENT PROPOSED IMPROVEMENT OF INFO RM ATICS READING COMPREHENSION TESTS FOR HANOI UNIVERSITY OF