Designing & evaluating an English reading test for the non-majors of Civil Engineering at Haiphong private university

Designing & evaluating an English reading test for the non-majors of Civil Engineering at Haiphong private university

Vietnam national university, hanoiCollege of foreign languages

- -Designing & evaluating an English reading test for the non-majors of Civil Engineering

at Haiphong private university

Thiết kế và đánh giá một bài kiểm tra tiếng anh chuyên ngànhcho sinh viên xây dựng dân dụng tại

trờng đại học dân lập hải phòngM.A minor thesis

Field: methodologyCode: 50702

Course: k11

Supervisor : Tran Hoai Phuong, MEd.

Hanoi - August 2005

During the process of further studying and conducting this research I was reallyhonored to receive guidance, assistance, and encouragement from various lecturers as well assupervisors among whom I would like to acknowledge my sincere thanks to the leaders of theCollege of Foreign Languages who have given me permission and created favorable conditionsfor study and research.

I would also like to thank my supervisor, Mrs.Tran Hoai Phuong, Med, who really

sympathized with me and also gave me great help as well as invaluable guidance andencouragement from the very start to the end of my research.

It is also my pleasure to give my special thanks to the students of classes XD 501, XD502 and XD 503 at Hai Phong Private University who enthusiastically took part in doing thetest and helped me collect the results of the test.

I also benefited greatly from talks and discussions with my colleagues so let me thankall of them for what they have directly or indirectly contributed.

And finally I really want to thank my beloved husband who always gives great supportto my further study.

Nguyen Thi Phuong Thu

List of abbreviations

1 HPU Haiphong Public University

2 CE Civil Engineering

3 CEE Civil Engineering English

4 ESP English for Specific Purposes

5 MCQ Multiple Choice Question

10 N The number of the scores

11 x The raw score

12 f The frequency with which a score occurs

13 H The highest value

14 L The lowest value

23 CU The number of the correct asnwers of the upper half

24 CL The number of the correct asnwers of the lower half

25 gd good discrimination

26 md bad discrimination

27 bi bad item

28 p Spearman rho correlation coefficient

29 SU Score on the upper half

30 SL Score on the lower half

Table of contents

AcknowledgementList of abbreviationsPart I: Introduction1.Rationale

2.Aims of the study 3.Scope of the study4.Methods of the study5.Design of the studyPart II: Development

Chapter one: Literature review

1.3.4.Selective deletion gap filling1.3.5.C tests

1.3.6.Coloze elide

1.3.7.Information transfer1.3.8.Jumbled sentences1.3.9.Matching

1.3.10.Jumbled paragraphs

1.4.Major characteristics of a good test

1.41.Reliability 1.4.2.Validity

1.4.2.1.Content validity1.4.2.2.Face validity

1.4.2.3.Criterion-related validity1.4.2.4.Construct validity

1.5.Achievement tests

1.5.1.Class progress test1.5.2.Final achievement test

Summary

Chapter two: Methodology

2.1.A quantitative study

2.2.The selection of participants2.3.The materials

2.4.Methods of data collection and data analysis2.5.Limitations of the research

Chapter three: Discussion

3.1-The content area of the test

3.2-The relative weights of the different parts of the test3.3-Constructing the test

3.4-Administering the test3.5-Marking the test

3.6-Test scores interpreting and evaluation

3.6.1.The frequency distribution3.6.2.The central tendency

3.6.2.1.The mode3.6.2.2.The median3.6.2.3.The mean3.6.3.The dispersion

3.6.3.1.The low-high3.6.3.2.The range

3.6.3.3.The standard deviation

3.7-Test item analysis and evaluation

3.7.1.Item difficulty3.7.2.Item discrimination

3.8.Estimating reliability

Part III: Conclusion and recommendations

ReferencesAppendices

Part I: Introduction

Testing is a matter of concern to all teachers - whether we are in the classroom orengaged in syllabus/ materials, administration or research We know quite well that good testscan improve our teaching and stimulate student learning Although we may not want tobecome a measurement expert we may have to periodically evaluate student performances andprepare reports on student progress.

Haiphong Private University (HPU) is a university in which there are a number ofclasses of Civil Engineering (CE) for students of Construction Department Generallyspeaking, non-majors, especially the students of this department, lack background knowledgeof English The non-majors of CE have chances to learn General English (GE) during theirfirst three terms to prepare for their 120 periods of English for Specific Purposes (ESP) in thefourth term In fact, this type of English is quite demanding for them and many had to admitthat they could not learn it well As a result, many students failed after each final examination The causes for the above situation are various It might be because some students areeither too hesitant or too lazy to learn anew subject It might also be because some studentscould not overcome the difficulties they usually meet during their study, for example their ESPis too new or too demanding for them, or they have to learn many periods per week to leavetime for other subjects However, the reason which is no less important and which needs takinginto account is the matter of testing In general, teachers at HPU are well-qualified and whenteaching they are quite enthusiastic with good teaching methodology However, the results oftheir students’ tests are not always satisfactory, the scores they gained were often lower thanexpected Moreover, we teachers cannot deny the fact that sometimes the test results do notaccurately reflect the testees’ language competence

According to Brown (1994a: 373) and Hughes (1989: 1) “A great deal of languagetesting is of very poor quality Too often language testing has a harmful effect on teaching andlearning and too often they fail to measure accurately whatever it is they are intended tomeasure.”

For all the above reasons the author of this research study would like to take this

opportunity to undertake the study entitled “Designing a reading test for the non-majors ofCivil Engineering at Haiphong Public University” with a view to evaluating the students’

reading ability after one term’s study last school year (2004-2005) as well as to gaining someknowledge and experience of foreign language testing for herself after completing the study

2.Aims of the study

The minor thesis is aimed at designing an achievement test of ESP reading whichwould be conducted in a class of Civil Engineering English at HPU The test was considered as

a final examination Then the results of the test will be analysed, evaluated, and interpreted.The test takers are non - English - majors.

The specific aims of the research are:

 to assess the learners’ achievement in improving reading skill with English of CivilEngineering after 120 period reading course.

 to measure their aptitude for the reading skill.

 to diagnose their strength and weakness in reading the subject matter.

 to find out whether or not the test satisfies the qualities of a good test From therethe test will measure the effectiveness of the teacher’s teaching If the test is not agood one, some suggestions will be made for a better test form.

3.Scope of the study

“Not all language tests are of the same kinds They differ with respect to how they aredesigned, and what they are for; in other words, in respect to test method and test purpose.”

(Mc Namara, 2000: 5) For example, in terms of method, there are paper-and-pencil languagetests, performance tests, ect And in terms of purpose, there are achievement tests, proficiencytest, and so on In fact, the same form of test may be used for different purposes, although inother cases the purpose may affect the form.

Due to the limitation of time and ability, it is impossible for the author to design testsof all these types or of all the four language skills (speaking, writing, listening and reading).Therefore, this minor thesis is limited to designing and evaluating an achievement test of ESPreading for the non-majors at HPU and the reading tested was for communicative purposes.

4.Methods of the study

In this minor thesis the author designed an achievement test of reading, administered itand then evaluated it, so the method adopted is quantitative The data will be collected throughtesting the students’ reading ability of Civil Engineering English.

5.Design of the study

The study is composed of three parts:

*Part I is the presentation of basic information such as the rationale, the scope of the study,

the aims of the study, the methods of the study and finally the design of the study.

*Part II includes three chapters:

+ Chapter one is the literature review in which the literature that is related to language

testing and major characteristics of a good reading test is presented.

+ Chapter two is concerned with research methodologies including the methods

adopted in doing the research, the selection of participants, the materials, the methods of datacollection and data analysis.

+ Chapter three is the discussion, which is the main part of the study This chapter

reviews how a reading test of Civil Engineering for the non-majors at HPU was designed,administered, and then evaluated.

*Part III includes the conclusion and recommendations for further research on the topic

Following these parts are the references and appendices.

Part II : Development

Chapter one : Literature review

This chapter will provide an overview of the theoretical background of the research Itis composed of five small sections Section 1.1 brings a significant insight into the concept oflanguage testing Section 1.2 is the introduction of communicative language tests Testingreading skills will be discussed in section 1.3 which is followed by section 1.4 with theinvestigation into major characteristics of a good test The final area to be mentioned is a briefreview of achievement tests which is presented in section 1.5.

1.1.Language testing

An understanding of language testing is relevant both to those who are actuallyinvolved in creating language tests, and also to those who are involved in using tests or theinformation tests provide in practical research contexts For this very reason, this sectionwishes to take a close look at what a language test is.

Most researchers agree that language tests play many important roles in life Firstly themoment one does a test can be considered an important transitional moment in his life, forexample, a pupil wishing to enter a university has to pass the entrance tests, or a job seeker hasto do a certain test so that the employer will know whether he is competent, or if somebodyneeds to drive a motor or a car, he or she has to pass a driving test, ect Secondly, languagetests are also important to many occupations We teachers rarely teach without testing ourstudents’ performance in the subjects Tests will help us to put them in right places; therefore,language tests, if used properly, can be considered a valuable teaching device for any teacher,and they will contribute positively to the development of both teachers and learners Last butnot least, any researcher who needs measurement of the language proficiency of the subjectscannot do it without using an already existing test or designing his or her own test.

As for Caroll (1968) a test in general will certainly tell something about a testee’scharacteristics Thanks to the results from his test, it is possible for a teacher to judge whetherthis student is good or bad at the subject tested Caroll provides the following definition of a

test: “a psychological or educational test is a procedure designed to elicit certain behaviorfrom which one can make inferences about certain characteristics of an individual.” (Caroll,

 Placement tests

 Aptitude or Prognostic tests

 Direct tests versus indirect tests-Discrete- point tests versus intergrativetests

 Norm-referenced tests versus criterion-referenced tests Objective tests versus Subjective tests

 Communicative tests

Generally there are some approaches to tests, for example the essay-translationapproach, the structuralist approach, the integrative approach, or the communicative approach.However, in this minor thesis, I would like to choose only the communicative approach to

testing This approach focuses on how the language is used in communication (‘meaning’rather than ‘form’) This attempts to obtain different profiles of a learner’s performance in the

language

The development and the use of language tests involve an understandingof the natureof communicative language use and language ability, on the one hand, and of measurementtheory, on the other Each of these areas is complex in its own right.

In short, like teaching, testing is important to any teacher as well as for any student Itis difficult to deny that testing cannot be separated from teaching, testing can even be seen partof teaching Therefore, we teachers should pay great attention to the issue of testing in ourteaching.

1.2.Communicative language tests

There is one thing that is essential to the activities of designing a test and interpretingthe meaning of test scores It is the view of language and language use embodied in the test.

The term ‘test construct’ refers to these aspects of knowledge or skill possessed by the

candidate which are being measured To define test construct it is important to be clear aboutwhat knowledge of language consists of and how that knowledge is used in actual performance(i.e language use) It is also essential to understand what view the test takes of language usebecause if the view the test takes is different, then the test will be different As a result, thereporting of score will be different, and the test performance will be interpreted differently.Therefore, the difference of format between tests is not just incidental; it implies a differencebetween views of language and language use Accordingly, communicative language tests aredifferent from other types of tests such as discrete point test or integrative and pragmatic testsin the following aspects:

According to Mc Namara (2000: 17) discrete point test focuses on students’ knowledgeof the grammatical system, of vocabulary and aspects of pronunciation and tends to test theseaspects of knowledge in isolation With this type of test, multiple choice questions are mostsuitable This discrete point tradition of testing is seen as focusing too much on knowledge ofthe formal linguistic system for its own sake rather than on the way the knowledge is used toachieve communication.

Aslo as for Mc Namara using integrated tests is a new orientation in which integratedknowledge of relevant systemic features of language (pronunciation, grammar, vocabulary)with an understanding of context is deployed Yet, these tests are regarded as time consumingand difficult to score For example for an oral interview, the test will involve comprehension ofextended discourse (both spoken and written), and as a result besides the disadvantagesmentioned above it also requires trained raters

Because of those disadvantages another type of test, pragmatic test, replaced the oldones It focuses less on knowledge of language and more on psycholinguistic processinginvolved in language use With this type, a cloze test was seen the most suitable and was oncebelieved to be easy to construct, relatively easy to score However, it soon turned out to bemeasuring the same kinds of things as discrete point tests of grammar and vocabulary It alsofailed to test communicative skills.

In the early 1970s thanks to Hyme’s theory of communicative competence (anunderstanding of language and the ability to use language in context, particularly in terms ofthe social demand of performance, i.e knowing a language is more than knowing its rules ofgrammar) communicative language tests developed and it has the two following features:

’They are performance tests which require assessment to be carried out whenthe candidate is engaged in communication, either receptive or productive, or both.

They see language as a sociological phenomenon, focusing on the external,social functions of language while integrative and pragmatic tests see language as an internalphenomenon With this test, the use of authentic texts and real world tasks may be developed.’

(Mc Namara, 2000: 16).

One of its distinguishing feature that supersedes other types of tests is that besides systemic

features of language, it requires students’ careful study of the communicative roles and tasks.All the reasons discussed above are regarded as a strong impetus that initiates this minor thesisinto designing a reading test of ESP for communicative purpose, i.e it is a communicativelanguage test.

1.3-Testing reading skills

In a reading test, test items are often set basing on the text itself And often within thesame test more than one typed of item, maybe two, three or more types of the following itemsare used:

1.3.1 Multiple-choice questions (MCQs)

This is one of the most popularly used types for setting a reading comprehension test.When doing this test the candidate is required to select the answer from a number of givenoptions, only one of which is correct The marking is totally objective Selecting and settingitems are, however, subjective processes, and the decision about which is the correct answer isa matter of subjective judgment on the part of the item writer.

1.3.2 Short answer questions

In the test there are questions which require the candidates to write down specificanswers in spaces provided on the question paper.

1.3.3 Cloze

This type is also familiar with students In the cloze procedure, words are deleted froma text after allowing a few sentences of introduction The deletion rate is mechanically set,usually between every fifth and eleventh word because deleting too many or too few words cancause problems with test validity Candidates have to fill each gap by supplying the word theythink has been deleted

1.3.4 Selective deletion gap filling

It is selecting items for deletion based upon what is known about language, aboutdifficulty in text and about the way language works in a particular text.

1.3.5 C-Tests

In C-test every second word in a text is partially deleted In an attempt to ensuresolutions, students are given the first half of the deleted word The examinee completes theword on the test paper and an exact word scoring procedure is adopted.

1.3.8.Jumbled sentences

This type of test is intended to test the student’s understanding of a sequence of stagesin a process or events in a narrative A successful student is the one who can reorder jumbledsentences or unscrambled sentences of a story correctly.

Like MCQ test, matching is a familiar type of testing reading comprehension With thistest, candidates are required to identify the relationships between a list of entries in onecolumn with a list of responses in another column Candidates may have to match word withword, sentences with sentence, picture with sentence, etc.

1.3.10.Jumbled paragraphs

Similar to tasks involving jumbled sentences, test tasks with jumbled paragraphsrequire students to rearrange the given paragraphs in the correct order To do this studentshave to read through these paragraphs to get the main idea of the whole text In short, fortesting reading abilities different methods have been recommended and a teacher may use thisone or that one depending on certain purposes For example, to develop the communicativenature of tests the use of short answer questions, selective gap filling, C-tests, informationtransfer techniques or other restricted response formats are often preferred.

1.4 Major characteristics of language tests

Tests can serve pedagogical purpose, to be sure The most important consideration indesigning a language test is its usefulness This can be defined in terms of their qualities suchas reliability, validity, practicality, interactiveness, impact, or authenticity, etc Among thesethe four qualities which will be discussed below are more critical for good tests.

1.4.1 Reliability

Reliability is apparently an essential quality of test values; if the scores of a test are notrelatively consistent, they fail to provide us with the information about the ability we want tomeasure Reliability is considered a fundamental criterion against which any language test hasto be judged

‘Reliability is often defined as consistency of measurement’ (Bachman & Palmer,

1996:19) A reliable test score will be consistent across different characteristics of the testingsituation Thus, reliability can be considered to be a function of the consistency of scores fromone set of test tasks to another Or in other words, tests should not be plastic in theirmeasurements: if a student takes a test at the beginning of the course and again at the end, anyimprovement in his score should be the results of differences in his skills and not inaccuraciesin the test In the same way, it is important that the student’s score should be the same (or asnearly the same as possible) whether he takes one version of the test or another and whether

one person marks the test or another Reliability also means ‘the consistency with which a testmeasures the same thing all the time’(Harrison, 1987) This can be presented in the figure

- take enough samples of behavior,

- do not allow candidates too much freedom in choosing what and how to answer,- write unambiguous items,

- provide clear and explicit instructions,

- ensure that tests are well laid out and perfectly legible,

Scores on test tasks withcharacteristics A

Scores on test tasks with characteristics A’

- make sure candidates are familiar with format and testing techniques,- provide uniform and non-distracting conditions of administration,- use items that permit scoring which is as objective as possible,- make comparisons between candidates as direct as possible,- provide a detailed scoring key,

- train scorers,

- agree on acceptable responses and appropriate scores at outset of scoring,- identify candidates by number, not name, and

- employ multiple, independent scoring (Hughes, 1989: 36-42)

The concept of reliability is particularly important when considering language testswithinthe communicative paradigm (Porter, 1983) Davies (1965: 14) also shares the same

view but he also admits that ‘reliability is the first essential for any test; but for certain kindsof language test may be very difficult to achieve.’

1.4.2 Validity

The second quality that affects test usefulness is validity A test is said to be valid if itmeasures what it is intended to measure Or in other words, the test may be valid for somepurposes, but not for others For example, if the purpose of a test is to test ability tocommunicate in a foreign language, then it is valid if it actually tests ability to communicate Ifthe test is full of questions of grammar, then the test cannot be considered valid Moreover, if atest is to test reading ability, but it also tests writing, for example, then the test fails to have thevalidity for testing reading.

However, it is impossible to say whether a test is valid or not valid at all because thereare degrees of test validity, i.e this test may be more valid than that one Therefore, Moore

(1992) defined validity as “the degree to which a test measures what it is supposed tomeasure” There are different types of validity such as content, face, construct, criterion-

related validity, and they will be all discussed below.

1.4.2.1.Content validity

Among different types of validity, content validity is said to be the most important one,

but it is also the simplest “A test is said to have content validity if its content constitutes arepresentative sample of the language skills, structures, etc with which it means to beconcerned.” (Hughes, 1989: 22) In order to judge whether or not a test has content validity,

we need a specification of all the related aspects that the test is meant to cover, including the

skills or the structures Such a specification should be made at a very early stage in testconstruction.

According to Weir (1990: 24) the more a test stimulates the dimensions of observableperformance and accords with what is known about that performance, the more likely it is tohave content validity and construct validity Thus, for Kelly (1978: 8) content validity seems

‘an almost completely overlapping concept” with construct validity, and for Moller (1982: 68):‘the distinction between construct and content validity in language testing is not always verymarked, particularly for tests of general language proficiency.’ Slightly different from otherresearchers, Anastasi (1982: 131) defined content validity as: ‘essentially the systematicexamination of the test content to determine whether it covers a representative sample of thebehavior domain to be measured.’

So we could see that content validity has been defined differently, but most researchersagree that content validity is highly important for the two following reasons First, the greater atest’s content validity is, the more likely it is to be an accurate measure of what it is supposedto measure A test in which major areas identified in the specification are under-representedornot represented at all is unlikely to be accurate Secondly, such a test is likely to have harmfulbackwash effect Areas which are not tested are likely to become areas ignored in teaching andlearning.

1.4.2.2 Face validity

A test is said to have face validity if it looks as if it measures what it is supposed tomeasure Face validity is hardly a scientific concept, yet it is very important A test which doesnot have face validity may not be accepted by candidates, teachers, education authorities oremployers.

1.4.2.3 Criterion-related validity

There are essentially two kinds of criterion-related validity: concurrent validity andpredictive validity According to Viete (1992), concurrent validity is used to refer to therelationship between the test results and the results of another assessment (using anappropriate, reliable and validated assessment procedure) which was made at approximatelythe same time And predictive validity concerns the degree to which a test can predictcandidate’s future performance.

1.4.2.4 Construct validity

Like reliability, construct validity is essential to the usefulness of any language test.The term construct validity is used to refer to the extent to which we can interpret a given testscore as an indicator of the ability(ies) or construct(s), we want to measure The purpose ofconstruct validation is to provide evidence that underlying theoretical constructs being

measured are themselves valid Typically, construct validation begins with a psychologicalconstruct that is part of a formal theory The theory enables certain predictions about theconstruct variable will behave or be influenced under specified conditions The construct isthen tested under the conditions specified If the hypothesized results occur, the hypotheses aresupported and the construct is said to be valid Often this will involve a series of tests under avariety of conditions.

Test validity is the one that is always paid the most attention to since it is anindispensable quality of all good tests When constructing a test, the first thing to be focusedon is test validity Hughes (1989: 22) agrees that if in a test important parts are not defined or

not presented, it will fail to be accurate He notes that “the greater a test's content validity is,the more likely it is to be an accurate measure of what it is to measure.”

1.4.3 Practicality

Another quality of a good test which should not be forgotten is its practicality.Although it is different in nature from other qualities, practicality is not less important Unlikereliability and validity, practicality does not pertain to the uses that are made of test scores, butprimarily to the ways in which the test will be implemented in a given situation, and to whetherthe test will be developed and used at all Practicality often affects a tester’s decisions duringthe development of a test, i.e., at every stage of his testing.

Practicality can be defined as ‘the relationship between the resources that will berequired in the design, development, and use of the test and the resources that will beavailable for these activities’ (Bachman & Palmer, 1996: 35) This relationship can be

represented as in the figure below:

Available resourcesPracticality=

the extent of discrimination varies according to each kind of test For instance, an achievementtest should result in a wide range of scores because it is easier to make decisions about whereto separate one group of students from another so that they can be awarded different grades Adiagnostic test, however, may be intended to show that nearly all students have learnt thematerial tested, and in this case they should all get fairly high scores.

1.5 Achievement tests

Different researchers have different points of view of an achievement test According

to Harrison (1983: 65) ‘designing and setting an achievement test is a bigger and more formaloperation than the equivalent work for a diagnostic test, because the student's result is treatedas a qualification which has a particular value in relation to the results of other students Anachievement test involves more detailed preparation and covers a wide range of material, ofwhich only the sample can be assessed.’

Heaton (1988) defines achievement tests as the ones that are “based on what thestudents presumed to have learnt, not necessarily on what they have actually learnt nor onwhat has actually been taught.”

In Brown’s point of view “an achievement test is related directly to classroom lesson,units or even a total curriculum within a particular time frame.” (Brown, 1994: 259) In other

words, an achievement test measures a student’s mastery of what should have been taught It isthus concerned with covering a sample (or selection), which accurately represents the contentsof a syllabus or a course book Unlike progress test, achievement test should attempt to coveras much of the syllabus as possible If we confine our test to only part of the syllabus, thecontents of the test will not reflect all that the student has learnt.

Achievement test can be subdivided into class progress tests and final achievement tests.

1.5.1 The class progress test

The class progress test is often conducted during the course and is developed by theteacher himself after each chapter or each term He constructs such type of test to judge howsuccessful his teaching is and also to find out what his students have achieved from histeaching The class progress test is a teaching device and can be considered a good chance forthe students to prepare for the final achievement test.

1.5.2 The final achievement test

The final achievement test is more formal and intended to measure achievement on alarger scale (annual exams, entrance exams, final exams) The final achievement test is notwritten and administered by the teacher himself, but maybe by ministries of education, boardsof examiners, or by members of teaching institutions A final achievement test is often basedon an adopted syllabus and its approach, either syllabus-content approach or syllabus-objective

approach If the test is based on the former, its contents should be based directly on a coursesyllabus or on the textbooks and other materials chosen If it is based on the latter, its contentsare based directly on the objectives of the course.

In this chapter I have briefly dealt with the concept of a language test, how it is definedand what is important in designing it Moreover, I also mentioned the concept ofcommunicative language ability in which communicative competence was also discussed.Also, in this chapter the definition of an achievement test as well as testing reading skills werepresented because they play an important role in the process of doing this research.

Chapter two: Methodology

This chapter will include a brief introduction of a quantitative study, the selection ofparticipants who took part in doing the test, and the materials from which the test items weretaken The methods of data collection and data analysis are presented afterwards Finally comethe limitations of the research.

2.1.A quantitative study

Like qualitative research, quantitative research comes in many approaches includingdescriptive, correlational, exploratory, quasi-experimental, and true-experimental techniques.

As a teacher of Civil Engineering English, I designed this reading test to understandbetter how things are really operating in my own classroom as well as to describe theperformance of my learners in the reading skill After 120 period reading course 50 studentswere chosen from three different classes (XD501, XD 502, XD 503) to do a reading test in thetime given (60 minutes) and then the results collected from the testing papers would bedescribed in different terms with the use of the descriptive statistics technique Thecorrelational research technique was also used to find out the reliability coefficient latter in thestudy.

2.2.The selection of Participants

The students at Haiphong Private University mainly come from different towns andcities in the North of Vietnam They are generally aged between 18 and 22, or older.

At the university, they study for eight terms in four years There students are classifiedinto majors and non-majors of English The latter usually have to learn a foreign language, inthis case English, in only two years of their whole student lifeIn the first three terms, theystudy General English and in the fourth term English for Specific Purposes (ESP) After twoyears’ English learning, they are required to be able to read and translate their ESP atintermediate level However, students often have varying English levels prior to the course dueto the fact that at secondary school they learned different languages, including Russian,French, and Chinese It is therefore important for teachers to apply appropriate methods inteaching them GE as well as ESP to help them become more proficient It is also critical thatteachers give them suitable tests which meet their need and the requirements of society at thesame time.

2.3.The Materials

During the first three terms the CE students are required to learn all the 15 units inElementary Headway, and the first 8 units in Pre-intermediate Headway These three terms

include 205 periods in all, 75 periods for each term In the fourth term, they study 120 periodsof ESP usig a the 15-unit textbook on English for Civil Engineering.

2.4.Methods of data collection and data analysis

To collect data for the research, a 34-item test of Civil Engineering English readingwas delivered to 50 students of the Construction Department These non-majors did the testwithin the time frame given (60 minutes) Then the test papers were collected, and then weremarked, analysed, and interpreted Doing these things did point out how many students did thetest well, how many performed badly, the most frequent scores the testees got, how thesescores ranged, how many scores deviated from the mean, etc.

2.5.Limitations of the research

Like in any other studies, some limitations cannot be avoided in this one Firstly,because of the limitation of time as well as of ability, the author could design only one readingtest to be conducted on 50 students, which might not be a large number Yet, it is hoped thatthe results could be reliable and valid enough for the researcher to make inferences and cometo certain conclusions Secondly, instead of designing different types of test, the author wasable to make solely one type, that is an achievement test to measure the progress her studentshad made in terms of reading skills after undertaking the course of English for CivilEngineering in their last term in 2004-2005 school year From the results, the author could alsomeasure the effectiveness of her teaching

This chapter gives a brief account on a quantitative study, in which the author used thedescriptive statistics and correlational technique to analyse the data Following the methods,the selection of participants and materials has also been dealt with A quick introduction of thedata collection and data analysis methods was also presented and finally came the limitationsof the research.

Chapter three: Discussion

This chapter is the discussions firstly about the content area of the test, how the test wasdivided, how to construct and mark the test Afterwards, the whole test results and each testitem would be analysed and then interpreted Finally, the author will evaluate the test based onthe four criteria of a good test as mentioned in the previous chapter.

3.1 The content area of the test

The following topic checklist of the course book will help to point out the content area of thereading test.

The Topic checklist of the course book

Architectural compositionSkeleton constructionConcrete, reinforced concrete, prestressed

concreteUltimate carrying capacityand factor of safety

Pre-cast productsBreakwaters

Conveying, placing, compacting, and curingConcrete and strength testAsphalt concrete

Materials and propertiesStructure

Actions and sequencesArch and arch beam bridgesShear forces and bendingmoments in beams

Matrix methods in thecalculation of structure

English of Civil

Engineering Unit 1-p.1Unit 2-p.6Unit 3-p.10

Unit 4-p.14Unit 5-p.20Unit 6-p.24

Unit7-p.1 (Book 2)Unit 8-p.8

Unit 9-p.12Unit 10-p.20Unit 11-p.26Unit 12-p.29Unit 13-p.32Unit 14-p.35Unit 15-p.35Unit 16-p.42

The hinge Unit 17-p.46

3.2.The relative weights of the different parts of the test

The test is composed of 5 parts, and the weighting of each part is illustrated in the followingtable:

Test of reading

1 Factual text,approx.120 words

3.3.Constructing the test

To construct the reading test used in this research, the author went through thefollowing procedures:

Statement of the problem

There was a need for this achievement test to be administered at the end of the courseof training in the reading of Civil Engineering English (the students are graduates) The testwas intended to find out what progress was being made after 120 period study and also whatwere the greatest difficulties in learning that the students still had at the end of the course.Thanks to that future courses may give more attention in these areas Backwash is consideredimportant; the test should encourage the practice of the reading skills that the students need intheir university study The time allowed was one hour.

Types of text: The academic texts were from the course book entitled ‘English for CivilEngineering’ One sample text is provided in Appendix 1.

Addressees: Non-native speaker university students at HPU, or more specifically non-majors

of CE at HPU.

Topics: The topics were suitable for the candidates and the type of test, and the subject area

were neutral.

Operation: The test has 5 tasks and the candidates had to scan to locate specific information,

to match words/ phrases to make correct statements, to arrange words/ phrases to makecomplete sentences, to decide whether the given statements are true or false, and finally to fillblanks with the given words.

FORMAT AND TIMING

Scanning: 1 passage with about 120 words in length.

5 short answer items, the items in the order in which relevant informationappears in the texts Responses were controlled.

Time: 10 minutes.Detailed reading

-5 columns of words Responses were controlled.

CRITICAL LEVELS OF PERFORMANCE

All test items were written such that any student completing the course successfully would be

able to respond correctly to all of them Allowing for ‘performance errors’ on the part of

candidates, a critical level of 80 percent was set The students reaching this level would be theones succeeding in terms of the course’s objectives.

ITEM WRITING AND MODERATION

All the items in the test were based on a consideration of what a competent non-major wouldbe able to obtain from the texts Considerable time was set side for moderation and rewritingof items.

KEY