VIETNAM NATIONAL UNIVERSITY, HANOI COLLEGE OF FOREIGN LANGUAGES
DESIGNING & EVALUATING AN ENGLISH READING TEST FOR THE NON-MAJORS OF CIVIL ENGINEERING
AT HAIPHONG PRIVATE UNIVERSITY
THIẾT KẾ VA ĐÁNH GIÁ MỘT BÀI KIEM TRA TIENG ANH CHUYEN NGANH CHO SINH VIEN XAY DUNG DAN DUNG TAI
TRUONG DAI HOC DAN LAP HAI PHONG
M.A MINOR THESIS FIELD: METHODOLOGY CODE: 50702
COURSE: K11
By : Nguyen Thi Phuong Thu Supervisor : Tran Hoai Phuong, MEd
Trang 2ACKNOWLEDGEMENTS
During the process of further studying and conducting this research I was really honored to receive guidance, assistance, and encouragement from various lecturers as well as supervisors among whom I would like to acknowledge my sincere thanks to the leaders of the College of Foreign Languages who have given me permission and created favorable conditions for study and research
T would also like to thank my supervisor, Mrs.Tran Hoai Phuong, Med, who really sympathized with me and also gave me great help as well as invaluable guidance and encouragement from the very start to the end of my research
It is also my pleasure to give my special thanks to the students of classes XD 501, XD 502 and XD 503 at Hai Phong Private University who enthusiastically took part in doing the test and helped me collect the results of the test
I also benefited greatly from talks and discussions with my colleagues so let me thank all of them for what they have directly or indirectly contributed
And finally I really want to thank my beloved husband who always gives great support to my further study
Trang 3LIST OF ABBREVIATIONS Haiphong Public University
Civil Engineering
Civil Engineering English English for Specific Purposes Multiple Choice Question True
False Mean Sum of
The number of the scores The raw score
The frequency with which a score occurs The highest value
The lowest value Standard Deviation Item difficulty The number of the correct answers very easy easy difficult very difficult litem discrimination
Trang 429 SU Score on the upper half
Trang 5TABLE OF CONTENTS ACKNOWLEDGEMENT LIST OF ABBREVIATIONS PART IE: INTRODUCTION 1 RATIONALE
-AIMS OF THE STUDY -SCOPE OF THE STUDY -METHODS OF THE STUDY -DESIGN OF THE STUDY PART II: DEVELOPMENT
CHAPTER ONE: LITERATURE REVIEW 1.1.Language testing
1.2.Communicative language tests 1.3.Testing reading skills
Trang 61.4.2.2.Face validity 1.4.2.3.Criterion-related validity 1.4.2.4.Construct validity 1.4.3.Practicality 1.4.4.Discrimination 1.5.Achievement tests
1.5.1.Class progress test 1.5.2.Final achievement test Summary
CHAPTER TWO: METHODOLOGY
2.1.A quantitative study
2.2.The selection of participants 2.3.The materials
2.4.Methods of data collection and data analysis 2.5.Limitations of the research
SUMMARY
CHAPTER THREE: DISCUSSION
3.1-The content area of the test
3.2-The relative weights of the different parts of the test 3.3-Constructing the test
3.4-Administering the test 3.5-Marking the test
3.6-Test scores interpreting and evaluation 3.6.1.The frequency distribution 3.6.2.The central tendency
Trang 73.6.3 The dispersion
3.6.3.1.The low-high 3.6.3.2.The range
Trang 8PARTI: INTRODUCTION 1.RATIONALE
Testing is a matter of concern to all teachers - whether we are in the classroom or engaged in syllabus/ materials, administration or research We know quite well that good tests can improve our teaching and stimulate student learning Although we may not want to become a measurement expert we may have to periodically evaluate student performances and prepare reports on student progress
Haiphong Private University (HPU) is a university in which there are a number of classes of Civil Engineering (CE) for students of Construction Department Generally speaking, non-majors, especially the students of this department, lack background knowledge of English The non-majors of CE have chances to learn General English (GE) during their first three terms to prepare for their 120 periods of English for Specific Purposes (ESP) in the fourth term In fact, this type of English is quite demanding for them and many had to admit that they could not learn it well As a result, many students failed after each final examination
The causes for the above situation are various It might be because some students are either too hesitant or too lazy to learn anew subject It might also be because some students could not overcome the difficulties they usually meet during their study, for example their ESP is too new or too demanding for them, or they have to learn many periods per week to leave time for other subjects However, the reason which is no less important and which needs taking into account is the matter of testing In general, teachers at HPU are well-qualified and when teaching they are quite enthusiastic with good teaching methodology However, the results of their students’ tests are not always satisfactory, the scores they gained were often lower than expected Moreover, we teachers cannot deny the fact that sometimes the test results do not accurately reflect the testees’ language competence
Trang 9For all the above reasons the author of this research study would like to take this opportunity to undertake the study entitled “Designing a reading test for the non-majors of Civil Engineering at Haiphong Public University” with a view to evaluating the students’ reading ability after one term’s study last school year (2004-2005) as well as to gaining some knowledge and experience of foreign language testing for herself after completing the study
2.AIMS OF THE STUDY
The minor thesis is aimed at designing an achievement test of ESP reading which would be conducted in a class of Civil Engineering English at HPU The test was considered as a final examination Then the results of the test will be analysed, evaluated, and interpreted The test takers are non - English - majors
The specific aims of the research are:
= to assess the learners’ achievement in improving reading skill with English of Civil Engineering after 120 period reading course
= to measure their aptitude for the reading skill
= to diagnose their strength and weakness in reading the subject matter
= to find out whether or not the test satisfies the qualities of a good test From there the test will measure the effectiveness of the teacher’s teaching If the test is not a good one, some suggestions will be made for a better test form
3.SCOPE OF THE STUDY
“Not all language tests are of the same kinds They differ with respect to how they are designed, and what they are for; in other words, in respect to test method and test purpose.” (Mc Namara, 2000: 5) For example, in terms of method, there are paper-and-pencil language tests, performance tests, ect And in terms of purpose, there are achievement tests, proficiency test, and so on In fact, the same form of test may be used for different purposes, although in other cases the purpose may affect the form
Trang 10Therefore, this minor thesis is limited to designing and evaluating an achievement test of ESP reading for the non-majors at HPU and the reading tested was for communicative purposes
4.METHODS OF THE STUDY
In this minor thesis the author designed an achievement test of reading, administered it and then evaluated it, so the method adopted is quantitative The data will be collected through testing the students’ reading ability of Civil Engineering English
5.DESIGN OF THE STUDY The study is composed of three parts:
*Part I is the presentation of basic information such as the rationale, the scope of the study, the aims of the study, the methods of the study and finally the design of the study
*Part I includes three chapters:
+ Chapter one is the literature review in which the literature that is related to language testing and major characteristics of a good reading test is presented
+ Chapter two is concerned with research methodologies including the methods adopted in doing the research, the selection of participants, the materials, the methods of data collection and data analysis
+ Chapter three is the discussion, which is the main part of the study This chapter reviews how a reading test of Civil Engineering for the non-majors at HPU was designed, administered, and then evaluated
Trang 11PART II: DEVELOPMENT
CHAPTER ONE : LITERATURE REVIEW
This chapter will provide an overview of the theoretical background of the research It is composed of five small sections Section 1.1 brings a significant insight into the concept of language testing Section 1.2 is the introduction of communicative language tests Testing reading skills will be discussed in section 1.3 which is followed by section 1.4 with the investigation into major characteristics of a good test The final area to be mentioned is a brief review of achievement tests which is presented in section 1.5
1.1 LANGUAGE TESTING
An understanding of language testing is relevant both to those who are actually involved in creating language tests, and also to those who are involved in using tests or the information tests provide in practical research contexts For this very reason, this section wishes to take a close look at what a language test is
Most researchers agree that language tests play many important roles in life Firstly the moment one does a test can be considered an important transitional moment in his life, for example, a pupil wishing to enter a university has to pass the entrance tests, or a job seeker has to do a certain test so that the employer will know whether he is competent, or if somebody needs to drive a motor or a car, he or she has to pass a driving test, ect Secondly, language tests are also important to many occupations We teachers rarely teach without testing our students’ performance in the subjects Tests will help us to put them in right places; therefore, language tests, if used properly, can be considered a valuable teaching device for any teacher, and they will contribute positively to the development of both teachers and learners Last but not least, any researcher who needs measurement of the language proficiency of the subjects cannot do it without using an already existing test or designing his or her own test
Trang 12test: “a psychological or educational test is a procedure designed to elicit certain behavior from which one can make inferences about certain characteristics of an individual.” (Caroll,
1968: 46)
According to Hughes (1989: 9), tests can be classified as follow: = Proficiency tests
= Achievement tests
e Class progress tests e Final achievement tests = Diagnostic tests
= Placement tests
= Aptitude or Prognostic tests
= Direct tests versus indirect tests-Discrete- point tests versus intergrative tests
= Norm-referenced tests versus criterion-referenced tests = Objective tests versus Subjective tests
= Communicative tests
Generally there are some approaches to tests, for example the essay-translation approach, the structuralist approach, the integrative approach, or the communicative approach However, in this minor thesis, I would like to choose only the communicative approach to testing This approach focuses on how the language is used in communication (‘meaning’ rather than ‘form’) This attempts to obtain different profiles of a learner’s performance in the language
The development and the use of language tests involve an understandingof the nature of communicative language use and language ability, on the one hand, and of measurement theory, on the other Each of these areas is complex in its own right
Trang 131.2.;COMMUNICATIVE LANGUAGE TESTS
There is one thing that is essential to the activities of designing a test and interpreting the meaning of test scores It is the view of language and language use embodied in the test The term ‘test construct’ refers to these aspects of knowledge or skill possessed by the candidate which are being measured To define test construct it is important to be clear about what knowledge of language consists of and how that knowledge is used in actual performance (i.e language use) It is also essential to understand what view the test takes of language use because if the view the test takes is different, then the test will be different As a result, the reporting of score will be different, and the test performance will be interpreted differently Therefore, the difference of format between tests is not just incidental; it implies a difference between views of language and language use Accordingly, communicative language tests are different from other types of tests such as discrete point test or integrative and pragmatic tests in the following aspects:
According to Mc Namara (2000: 17) discrete point test focuses on students’ knowledge of the grammatical system, of vocabulary and aspects of pronunciation and tends to test these aspects of knowledge in isolation With this type of test, multiple choice questions are most suitable This discrete point tradition of testing is seen as focusing too much on knowledge of the formal linguistic system for its own sake rather than on the way the knowledge is used to achieve communication
Aslo as for Mc Namara using integrated tests is a new orientation in which integrated knowledge of relevant systemic features of language (pronunciation, grammar, vocabulary) with an understanding of context is deployed Yet, these tests are regarded as time consuming and difficult to score For example for an oral interview, the test will involve comprehension of extended discourse (both spoken and written), and as a result besides the disadvantages mentioned above it also requires trained raters
Trang 14In the early 1970s thanks to Hyme’s theory of communicative competence (an understanding of language and the ability to use language in context, particularly in terms of the social demand of performance, i.e knowing a language is more than knowing its rules of grammar) communicative language tests developed and it has the two following features:
’They are performance tests which require assessment to be carried out when the candidate is engaged in communication, either receptive or productive, or both
They see language as a sociological phenomenon, focusing on the external, social functions of language while integrative and pragmatic tests see language as an internal phenomenon With this test, the use of authentic texts and real world tasks may be developed.’ (Mc Namara, 2000: 16)
One of its distinguishing feature that supersedes other types of tests is that besides systemic features of language, it requires students’ careful study of the communicative roles and tasks All the reasons discussed above are regarded as a strong impetus that initiates this minor thesis into designing a reading test of ESP for communicative purpose, i.e it is a communicative language test
1.3-TESTING READING SKILLS
In a reading test, test items are often set basing on the text itself And often within the same test more than one typed of item, maybe two, three or more types of the following items are used:
1.3.1 Multiple-choice questions (MCQs)
Trang 151.3.2 Short answer questions
In the test there are questions which require the candidates to write down specific answers in spaces provided on the question paper
1.3.3 Cloze
This type is also familiar with students In the cloze procedure, words are deleted from a text after allowing a few sentences of introduction The deletion rate is mechanically set, usually between every fifth and eleventh word because deleting too many or too few words can cause problems with test validity Candidates have to fill each gap by supplying the word they think has been deleted
1.3.4 Selective deletion gap filling
It is selecting items for deletion based upon what is known about language, about difficulty in text and about the way language works in a particular text
1.3.5 C-Tests
In C-test every second word in a text is partially deleted In an attempt to ensure solutions, students are given the first half of the deleted word The examinee completes the word on the test paper and an exact word scoring procedure is adopted
1.3.6 Cloze elide
In cloze elide test, words that do not belong to the original text are inserted into a reading passage and candidates have to indicate where these insertions have been made
1.3.7 Information transfer
Trang 161.3.8.Jumbled sentences
This type of test is intended to test the student’s understanding of a sequence of stages in a process or events in a narrative A successful student is the one who can reorder jumbled sentences or unscrambled sentences of a story correctly
1.3.9.Matching
Like MCQ test, matching is a familiar type of testing reading comprehension With this test, candidates are required to identify the relationships between a list of entries in one column with a list of responses in another column Candidates may have to match word with word, sentences with sentence, picture with sentence, etc
1.3.10.Jumbled paragraphs
Similar to tasks involving jumbled sentences, test tasks with jumbled paragraphs require students to rearrange the given paragraphs in the correct order To do this students have to read through these paragraphs to get the main idea of the whole text In short, for testing reading abilities different methods have been recommended and a teacher may use this one or that one depending on certain purposes For example, to develop the communicative nature of tests the use of short answer questions, selective gap filling, C-tests, information transfer techniques or other restricted response formats are often preferred
1.4 MAJOR CHARACTERISTICS OF LANGUAGE TESTS
Tests can serve pedagogical purpose, to be sure The most important consideration in designing a language test is its usefulness This can be defined in terms of their qualities such as reliability, validity, practicality, interactiveness, impact, or authenticity, etc Among these the four qualities which will be discussed below are more critical for good tests
1.4.1 Reliability
Trang 17measure Reliability is considered a fundamental criterion against which any language test has to be judged
‘Reliability is often defined as consistency of measurement’ (Bachman & Palmer, 1996:19) A reliable test score will be consistent across different characteristics of the testing situation Thus, reliability can be considered to be a function of the consistency of scores from one set of test tasks to another Or in other words, tests should not be plastic in their measurements: if a student takes a test at the beginning of the course and again at the end, any improvement in his score should be the results of differences in his skills and not inaccuracies in the test In the same way, it is important that the student’s score should be the same (or as nearly the same as possible) whether he takes one version of the test or another and whether one person marks the test or another Reliability also means ‘the consistency with which a test measures the same thing all the time’(Harrison, 1987) This can be presented in the figure below:
- Reliabili
Scores on test tasks with y Scores on test tasks
characteristics A with characteristics A’
Figure 1: Reliability
There are therefore three aspects to reliability: the circumstances in which the test is taken, the way in which it is marked, and the uniformity of the assessment it makes
According to Hughes (1989) there are two components of test reliability: the performance of candidates from occasion to occasion and the reliability of the scoring Therefore, to make tests more reliable Hughes (1989) gives a long list of and clear instructions for what we should do:
- take enough samples of behavior,
- do not allow candidates too much freedom in choosing what and how to answer, - write unambiguous items,
- provide clear and explicit instructions,
- ensure that tests are well laid out and perfectly legible,
Trang 18- provide a detailed scoring key, - train scorers,
- agree on acceptable responses and appropriate scores at outset of scoring, - identify candidates by number, not name, and
- employ multiple, independent scoring (Hughes, 1989: 36-42)
The concept of reliability is particularly important when considering language tests withinthe communicative paradigm (Porter, 1983) Davies (1965: 14) also shares the same view but he also admits that ‘reliability is the first essential for any test; but for certain kinds of language test may be very difficult to achieve.”
1.4.2 Validity
The second quality that affects test usefulness is validity A test is said to be valid if it measures what it is intended to measure Or in other words, the test may be valid for some purposes, but not for others For example, if the purpose of a test is to test ability to communicate in a foreign language, then it is valid if it actually tests ability to communicate If the test is full of questions of grammar, then the test cannot be considered valid Moreover, if a test is to test reading ability, but it also tests writing, for example, then the test fails to have the validity for testing reading
However, it is impossible to say whether a test is valid or not valid at all because there are degrees of test validity, i.e this test may be more valid than that one Therefore, Moore (1992) defined validity as “the degree to which a test measures what it is supposed to measure” There are different types of validity such as content, face, construct, criterion- related validity, and they will be all discussed below
1.4.2.1.Content validity
Trang 19skills or the structures Such a specification should be made at a very early stage in test construction
According to Weir (1990: 24) the more a test stimulates the dimensions of observable performance and accords with what is known about that performance, the more likely it is to have content validity and construct validity Thus, for Kelly (1978: 8) content validity seems ‘an almost completely overlapping concept” with construct validity, and for Moller (1982: 68): ‘the distinction between construct and content validity in language testing is not always very marked, particularly for tests of general language proficiency.’ Slightly different from other researchers, Anastasi (1982: 131) defined content validity as: ‘essentially the systematic examination of the test content to determine whether it covers a representative sample of the behavior domain to be measured.”
So we could see that content validity has been defined differently, but most researchers agree that content validity is highly important for the two following reasons First, the greater a test’s content validity is, the more likely it is to be an accurate measure of what it is supposed to measure A test in which major areas identified in the specification are under-representedor not represented at all is unlikely to be accurate Secondly, such a test is likely to have harmful backwash effect Areas which are not tested are likely to become areas ignored in teaching and learning
1.4.2.2 Face validity
A test is said to have face validity if it looks as if it measures what it is supposed to measure Face validity is hardly a scientific concept, yet it is very important A test which does not have face validity may not be accepted by candidates, teachers, education authorities or employers
1.4.2.3 Criterion-related validity
Trang 20the same time And predictive validity concerns the degree to which a test can predict candidate’s future performance
1.4.2.4 Construct validity
Like reliability, construct validity is essential to the usefulness of any language test The term construct validity is used to refer to the extent to which we can interpret a given test score as an indicator of the ability(ies) or construct(s), we want to measure The purpose of construct validation is to provide evidence that underlying theoretical constructs being measured are themselves valid Typically, construct validation begins with a psychological construct that is part of a formal theory The theory enables certain predictions about the construct variable will behave or be influenced under specified conditions The construct is then tested under the conditions specified If the hypothesized results occur, the hypotheses are supported and the construct is said to be valid Often this will involve a series of tests under a variety of conditions
Test validity is the one that is always paid the most attention to since it is an indispensable quality of all good tests When constructing a test, the first thing to be focused on is test validity Hughes (1989: 22) agrees that if in a test important parts are not defined or not presented, it will fail to be accurate He notes that “the greater a test's content validity is, the more likely it is to be an accurate measure of what it is to measure.”
1.4.3 Practicality
Another quality of a good test which should not be forgotten is its practicality Although it is different in nature from other qualities, practicality is not less important Unlike reliability and validity, practicality does not pertain to the uses that are made of test scores, but primarily to the ways in which the test will be implemented in a given situation, and to whether the test will be developed and used at all Practicality often affects a tester’s decisions during the development of a test, i-e., at every stage of his testing
Trang 21Available resources Practicality=
Required resources
When practicality = 1, the test development and use is practical When practicality< 1, the test development and use is not practical
In a nutshell, when designing a test the tester should always bare in mind this quality- practicality-to ensure that the test is as economical as possible, both in time (preparation, sitting and marking) and in cost (materials and hidden costs of time spent) In other words, a practical test is the one which can minimize the use of the available resources, i.e., the required resources must not be more than the available resources
1.4.4 Discrimination
Finally, a discussion of the basic concepts behind testing would be incomplete without the treatment of the closely related idea of discrimination According to Harrison (1994:14) discrimination is ‘the extent to which a test separates the students from each other.’ However, the extent of discrimination varies according to each kind of test For instance, an achievement test should result in a wide range of scores because it is easier to make decisions about where to separate one group of students from another so that they can be awarded different grades A diagnostic test, however, may be intended to show that nearly all students have learnt the material tested, and in this case they should all get fairly high scores
1.5 ACHIEVEMENT TESTS
Trang 22Heaton (1988) defines achievement tests as the ones that are “based on what the students presumed to have learnt, not necessarily on what they have actually learnt nor on what has actually been taught.”
In Brown’s point of view “an achievement test is related directly to classroom lesson, units or even a total curriculum within a particular time frame.” (Brown, 1994: 259) In other words, an achievement test measures a student’s mastery of what should have been taught It is thus concerned with covering a sample (or selection), which accurately represents the contents of a syllabus or a course book Unlike progress test, achievement test should attempt to cover as much of the syllabus as possible If we confine our test to only part of the syllabus, the contents of the test will not reflect all that the student has learnt
Achievement test can be subdivided into class progress tests and final achievement tests
1.5.1 The class progress test
The class progress test is often conducted during the course and is developed by the teacher himself after each chapter or each term He constructs such type of test to judge how successful his teaching is and also to find out what his students have achieved from his teaching The class progress test is a teaching device and can be considered a good chance for the students to prepare for the final achievement test
1.5.2 The final achievement test
Trang 23Summary
Trang 24CHAPTER TWO: METHODOLOGY
This chapter will include a brief introduction of a quantitative study, the selection of participants who took part in doing the test, and the materials from which the test items were taken The methods of data collection and data analysis are presented afterwards Finally come the limitations of the research
2.1.A QUANTITATIVE STUDY
Like qualitative research, quantitative research comes in many approaches including descriptive, correlational, exploratory, quasi-experimental, and true-experimental techniques
As a teacher of Civil Engineering English, I designed this reading test to understand better how things are really operating in my own classroom as well as to describe the performance of my learners in the reading skill After 120 period reading course 50 students were chosen from three different classes (XD501, XD 502, XD 503) to do a reading test in the time given (60 minutes) and then the results collected from the testing papers would be described in different terms with the use of the descriptive statistics technique The correlational research technique was also used to find out the reliability coefficient latter in the study
2.2 THE SELECTION OF PARTICIPANTS
The students at Haiphong Private University mainly come from different towns and cities in the North of Vietnam They are generally aged between 18 and 22, or older
Trang 25teaching them GE as well as ESP to help them become more proficient It is also critical that teachers give them suitable tests which meet their need and the requirements of society at the same time
2.3 THE MATERIALS
During the first three terms the CE students are required to learn all the 15 units in Elementary Headway, and the first 8 units in Pre-intermediate Headway These three terms include 205 periods in all, 75 periods for each term In the fourth term, they study 120 periods of ESP usig a the 15-unit textbook on English for Civil Engineering
2.4.METHODS OF DATA COLLECTION AND DATA ANALYSIS
To collect data for the research, a 34-item test of Civil Engineering English reading was delivered to 50 students of the Construction Department These non-majors did the test within the time frame given (60 minutes) Then the test papers were collected, and then were marked, analysed, and interpreted Doing these things did point out how many students did the test well, how many performed badly, the most frequent scores the testees got, how these scores ranged, how many scores deviated from the mean, etc
2.5.LIMITATIONS OF THE RESEARCH
Trang 26Summary
Trang 27CHAPTER THREE: DISCUSSION
This chapter is the discussions firstly about the content area of the test, how the test was divided, how to construct and mark the test Afterwards, the whole test results and each test item would be analysed and then interpreted Finally, the author will evaluate the test based on the four criteria of a good test as mentioned in the previous chapter
3.1 THE CONTENT AREA OF THE TEST
The following topic checklist of the course book will help to point out the content area of the reading test THE TOPIC CHECKLIST OF THE COURSE BOOK
Topic Material Number of unit/ page
Architectural composition Unit 1-p.1
Skeleton construction Unit 2-p.6
Concrete, reinforced Unit 3-p.10
concrete, prestressed concrete
Ultimate carrying capacity Unit 4-p.14
and factor of safety
Pre-cast products Unit 5-p.20
Breakwaters English of Civil ngineering Unit 6-p.24
Conveying, placing, Unit7-p.1 (Book 2)
compacting, and curing
Concrete and strength test Unit 8-p.8
Asphalt concrete Unit 9-p.12
Materials and properties Unit 10-p.20
Structure Unit 11-p.26
Trang 28
Actions and sequences Unit 13-p.32
Arch and arch beam bridges Unit 14-p.35
Shear forces and bending Unit 15-p.35 moments in beams Matrix methods in the Unit 16-p.42 calculation of structure The hinge Unit 17-p.46
3.2 THE RELATIVE WEIGHTS OF THE DIFFERENT PARTS OF THE TEST
The test is composed of 5 parts, and the weighting of each part is illustrated in the following table: Test of reading
Part Input Response/ Item type Scores Weighting Factual text, 5 comprehension 1 10 20% approx 120 words questions Matching to make 5 2 | 5 word columns 10 20% sentences 10 jumbled Rearrangin 3 J gins 10 20% sentences
4 10 statements True / False 10 20%
Factual text, approx
5 120 words with 5 Blank filling 10 20%
blanks
Trang 293.3.CONSTRUCTING THE TEST
To construct the reading test used in this research, the author went through the following procedures:
STATEMENT OF THE PROBLEM
There was a need for this achievement test to be administered at the end of the course of training in the reading of Civil Engineering English (the students are graduates) The test was intended to find out what progress was being made after 120 period study and also what were the greatest difficulties in learning that the students still had at the end of the course Thanks to that future courses may give more attention in these areas Backwash is considered important; the test should encourage the practice of the reading skills that the students need in their university study The time allowed was one hour
SPECIFICATIONS CONTENT
Types of text: The academic texts were from the course book entitled ‘English for Civil Engineering’ One sample text is provided in Appendix 1
Addressees: Non-native speaker university students at HPU, or more specifically non-majors
of CE at HPU
Topics: The topics were suitable for the candidates and the type of test, and the subject area were neutral
Operation: The test has 5 tasks and the candidates had to scan to locate specific information, to match words/ phrases to make correct statements, to arrange words/ phrases to make complete sentences, to decide whether the given statements are true or false, and finally to fill blanks with the given words
FORMAT AND TIMING
Scanning: 1 passage with about 120 words in length
Trang 30Time: 10 minutes Detailed reading -5 columns of words Responses were controlled Time: 10 minutes -10 jumbled sentences to be rearranged Responses were controlled Time: 20 minutes -10 statements to be marked T or F Responses were controlled Time: 10 minutes
-1 passage with about 120 words in length 5 gaps to be filled Responses were expected Time: 10 minutes
CRITICAL LEVELS OF PERFORMANCE
All test items were written such that any student completing the course successfully would be able to respond correctly to all of them Allowing for ‘performance errors’ on the part of candidates, a critical level of 80 percent was set The students reaching this level would be the ones succeeding in terms of the course’s objectives
SCORING PROCEDURES
There was a detailed key and the scoring was completely objective SAMPLING
The texts were chosen from a variety of topics in the course book Draft items were written before the test was officially used
ITEM WRITING AND MODERATION
All the items in the test were based on a consideration of what a competent non-major would be able to obtain from the texts Considerable time was set side for moderation and rewriting of items
KEY
Trang 31
HAIPHONG PUBLIC UNIVERSITY ACHIEVEMENT TEST
Tesfe@"S ƒHLÏÏ HŒITL€ s55 se se Seeeereeeexeseree Skill: READING
Mark: Level: INTERMEDIATE
Time allowed: 60 MINUTES
Question 1: Read the folloui t t ti iven bel
Conveying devices may be wheelbarrow, bottom dump bucket, dump truck If necessary concrete may be pumped through hoses and steel pipelines The mode of transport depends on the quality of concrete to be placed, the equipment available and other factors The method employed must prevent the separation of the materials, called segregation, and insure that concrete of good quality is deposited in the form
The forms are made of timber or metal of a size and shape suitable for the finished work They must be of sufficient strength and rigidity to support the wet material and allow it to be properly compacted They are so constructed that they may be easily removed when the concrete has hardened The interior of the forms must be oiled or soaped to prevent the concrete from adhering to the forms
Trang 32Question 2: Use th Js/_pk ; below t | t Jescribing th ties of material
Steel high tensile strength it can resist high compressive Stone has the | good sound isolation Thi forces
is
Glass wool | property good thermal isolation it can resist high tensile forces means
Brick of high compressive strength it does not transmit heat easily it does not transmit sound easily
TƠ
—
Đ TL TH HH TT TT TH HT HT TH TT Tà TT TH ren như
TL HH HH TT HH TH TT TT HH Tà TH Tà TT TH TT TT TT TH gu tà net
Trang 3316-be/ spread/ shall/ mixture/ by/ asphalt/ paver
statements are truce or false (Write T or F) 20-Glass wool is a heavy material
21-Rubber cannot be stretched or compressed 22-Concrete is a light material so it is easy to lift 23-We can burn wood because it is combustible 24-Mild steel can resist corrosion
25-Rubber is plastic while clay is elastic
26-Because copper is a good conductor of heat so heat can be easily transferred through it 27-We can easily scratch glass because it is soft
28-Concrete cannot be bunt because it is non-combustible 29-Stainless steel is corrosion-resistant
Question 5: Fill each blank below with ONE of the given words
multi-story minimum timber
architecture maximum possible
impossible low steel
Trang 34
" to build higher and higher, to meet today’s ever increasing demands The pattern of our large cities is being determined by skeleton structures of steel and concrete just as decisively as the pattern of medieval cities was determined by the (33) frame Widespread use has made the modern skeleton structure a central theme of contemporary (34)
3.4.ADMINISTERING THE TEST
In order to accomplish the two purposes of test administration for this reading test- collecting feedback to assess usefulness of the reading course and making inferences about test takers’ language ability-it is necessary to have some control over the procedures for administering it These involve guiding the test takers through the following process of taking the test:
Preparing the testing environment
Trang 35Communicating the instructions
‘The second step in administering the test was to give the instructions in such a way that they would be understood by all the test takers When administering the test it is essential that the test takers receive the full benefit of the instructions’ (Bachman and Palmer, 1981: 233) This included the obvious steps of providing suitable conditions (time, lighting, lack of distraction) for reading written instructions with the help of the two examiners
Maintaining a supportive environment
The next step is maintaining a supportive testing environment throughout the test This includes avoiding distractions due to temperature, noise excessive movement, etc
Collecting the tests
The final step in the test administration was collecting the tests.The testing papers were collected by the examiners after the allowed time in each testing room was over When they were being collected, the test takers left at their own peace
3.5 MARKING THE TEST
The testing papers were marked according to the band scores on the 0 — 10 scale as officially approved by the HPU board of examiners after they were collected
3.6 TEST SCORES INTERPRETING AND EVALUATION
The results of language tests are most often reported as numbers of scores, and it is these scores, ultimately, that test users will make use of The test scores of the 50 student participants were interpreted and analysed This very analysis will simply provide a summary of how the students did the test, and to check on the test’s reliability and to have some idea of how dependable the test scores were The following steps will provide the reader with an outline of how such analysis can be conducted
3.6.1.The frequency distribution:
Trang 36answers on a test The frequency distribution of the reading test that the author conducted is presented by the diagram below:
Raw marks on reading skill Number of students 1°15 2 25 3 35 4 45 5 55 6 65 7 7.5 8 85 9 Scores
(It is essential to remember that the total score of the test is 50, however, after the marking the total score each student got was divided by 5 to suit the 0-10 scale previously approved by the board of examiners)
The diagram above can be seen as self-explanatory: the vertical dimension indicates the number of candidates scoring within a particular range of scores; the horizontal dimension shows what these ranges are
When looking at the diagram, it is clearly seen that the students got different marks ranging from | to 9, i.e the lowest score was | and the highest was 9 The charts also tells that the set of scores was distributed quite unevenly, for example no student got marks 1.5, 2.5, 8.5; the score that most of the students got was 5.5 It also points out clearly the outcome (the students who got marks 5, 5.5, 6, 6.5, 7, 7.5, 8, and 9 would pass, and those getting marks under 5 would fail,) of the test
3.6.2.The central tendency
Trang 373.6.2.1.-The mode
The MODE is the value in a set of numbers that occurred most frequently In a way, the mode is the simplest of the three central tendency statistics discussed here because it requires no computation In this case the mode is 5.5 because it is the most frequent value
3.6.2.2.The median
The MEDIAN is the point in the distribution below which 50% of the values lie and above which 50% lie To find the median for this case, first place the values in order from low to high Then, examine the value above and below which 50% marks lie Here the median is 5
3.6.2.3.The mean
The most widely used measure of central tendency is the MEAN, which is more commonly called the AVERAGE The mean is the sum of all the values in a distribution divided by the total number of values (50)
Ds
N where: M=mean
The formula for the mean is: M =
X= sum of (or add up) N= the number of the scores x = the raw score
f =the frequency with which a score occurs
Trang 3855 12 = 66 6 4 = 24 6.5 1 = 65 7 6 = 42 15 4 = 30 8 3 = 24 8.5 0 = 0 9 1 = 9 Daf = 2675 Mee: 2765 5 «
From the above analysis we have the mean = 5.5 and the median = 5.As a result there’s a quite fairly correspondence between the mean and the median When comparing to the results the students got last terms it is possibly accepted because when studying General English the score they got after their exams were a little higher (the median and the mean generally ranged from 6 to 7) It is because of some reasons, firstly they had longer time to get in touch with the General English (at least 225 periods) Secondly this English was not so hard Therefore with the mean of 5.5 and the median of 5, the test results are quite satisfactory
3.6.3.The dispersion
Knowing about the central tendency of a set of numbers is a highly helpful way of characterizing the most typical behavior in a group It doesn’t, however, tell us anything about the way the numbers spread out around that central or typical behavior To know such a thing we need to find out the dispersion, which can be defined as ‘the degree to which the individual numbers vary away from the central tendency’ (Brown and Rodgers, 2002: 130) There are three primary ways of examining dispersion: the low-high, the range, and the standard deviation
3.6.3.1.The low-high
Trang 39high to low, we can see immediately that the lowest value was 1 and the highest value was 9 Thus, the low-high is 1-9
3.6.3.2.The range
Trang 403.6.3.3.The standard deviation (SD)
The best overall indicator of dispersion of the reading test is the STANDARD DEVIATION It is the degree to which the group of scores deviate from the mean Brown (1988: 69) defined it as ‘a sort of average of the differences of all scores from the mean’ The standard deviation is ‘a sort of average’ because you are averaging some values by adding them up and dividing by the number of values, just as you did in calculating the mean So the equation for the standard deviation starts with adding the squared difference between the value and the mean (5.5) up and dividing the number of the test takers (50):
SD = pe = {2a
where: SD: standard deviation X: values