Trang 1 THAI NGUYEN UNIVERSITY SCHOOL OF FOREIGN LANGUAGES DINH THI LIEN AN EXPLORATION INTO TEACHERS’ PERCEPTIONS AND PRACTICES IN DESIGNING ENGLISH ACHIEVEMENT TESTS Trang 2 THAI NGU
INTRODUCTION
The rationale of the study
It is widely accepted that English has now become the international lingua franca and is playing a more and more dominant role in people’s lives, ranging from politics, economics, and health care, to entertainment and education More specifically, many countries have changed their educational syllabi so that children are starting to learn English at a younger age
In line with globalization, English language teaching and learning is a critical issue in educational systems around the world, including in developing countries like Vietnam where English is considered as a foreign language In Vietnam, learners’ acquisition of English can be dramatically influenced by government policy toward the role of English in society Under these circumstances, English is taught as a compulsory subject in primary, junior, and senior high schools and as a complementary subject in universities and colleges As a direct result of this development, it is now necessary to teach and assess students’ English proficiency at the institutional level
According to Brown (1996), language tests are used to gather information about the learner’s language abilities in order to make informed educational decisions Thus, it is important to have good English tests to be able to make good decisions on learners’ English proficiency In the same view, Madsen (1983) holds the view that “good language tests help students learn the language by requiring them to study hard, emphasizing course objectives and showing them where they need to improve” (p.5) However, what makes a good test so that language teachers can make decisions about their classrooms based on the data it provides? Several researchers indicate that the characteristics of a good test must be valid, reliable, objective, and practical (Bachman, 1990; Bachman & Palmer, 1996; Brown, 1996; Henning, 1998)
Teaching and testing are inextricably linked components of the curriculum In Brown’s (1995) model of curriculum design, learning and teaching are the most important components of the curriculum Teachers are given a focus via tests that are drawn from instructional objectives As a result, tests provide feedback to teachers, students, and institutions
Achievement tests are those that provide direct feedback to teachers and students Teachers and students can use achievement tests to determine “the amount of language that each person is learning in a given period of time” (Brown, 1995, p.111) Thus, they support both parties in making judgments on the teaching-learning process Brown (1996) also points out that the achievement tests used in a language program may be adopted, developed, or invented Heaton suggests that “the best tests for the classroom are those you write yourself” (1990, p.23) because the teacher knows what learners need, what subjects have been covered in class, and how, which will help to maintain high content validity Furthermore, the language used in the classroom can be considered as a distinct social frameworrk that is specially designed for educational purposes to encourage language learning Obviously, teachers play a crucial role because they are the last users who implement the curriculum, have a thorough understanding of their students, and constantly engage with the students to support and motivate them to advance as planned in the curriculum at school As a result, all language teachers in the classrooms frequently conduct tests, not only to evaluate students’ development and performance but also to guide them in furthering their studies
On the contrary, there often have been a lot of gathered concerns in relation to the content validity of locally designed achievement English tests from the educators and administrators who always raise a question of whether the teacher-made English achievement tests faithfully reflect the course objectives, expected linguistic knowledge, skills as well as measures the real linguistic competence of test-takers Similar concerns also occur to the case of the teacher-made English achievement tests at a primary school in Thai Nguyen city, Vietnam In the light of these concerns, the researcher has been inspired and motivated to conduct the research on the teachers’ perceptions and practices with the title “An exploration into teachers’ perceptions and practices in designing English achievement tests at primary schools in Thai Nguyen city” with the belief that they will comprehend the reality of teacher-made
English achievement tests and provide them with the required teacher development programs to help them improve their proficiency in teaching and testing It might be assumed that, as a result of this procedure, teachers will have the opportunity to reflect on why they do and what they do when testing their students.
Statement of the problem
The perceptions of teachers about the tests employed in an institution are crucial since a difference in these opinions can do significant harm to instructional practices Tests should have a beneficial impact on teaching in order to improve language learning As Brown and Hudson (1998) say, “if the assessment procedures in a curriculum do not correspond to a curriculum’s goals and objectives, the tests are likely to create negative washback effects on those objectives and on the curriculum as a whole” (p 667-8) For learning to be supported, teaching objectives and tests should be matched Teachers may develop a negative attitude toward tests if there is a mismatch between teaching objectives and tests This may demotivate teachers, leading to learners “failure and institutions” failure As a result, it’s crucial to understand how teachers feel about the tests they use
Additionally, the function of the teacher-assessor in classroom assessment has received litmited attention in the English language teaching research literature (Davison, 2004) Recently, it has been reflected that teachers’ perceptions might not have always matched with what has been going on in their actual teaching context For instance, results from formal testing materials offered by public and private institutions appear to be the ones that students, parents, and even some teachers place the most attention on (Brown, 2004)
For these reasons, this study is being carried out in order to raise awareness of this potentially problematic practice and acquire a thorough understanding of the teacher-made achievement test.
Research sites
Teaching English as a foreign language in primary schools is gaining popularity throughout the world English has been an elective subject at the primary level in over 50 countries, 25 of which have made it as a compulsory subject (Jung &
Norton, 2002) The dominant role of English in current international communication increases the status of English teaching and learning in the educational system of Vietnam, and prompts English education to start at an early age (Van, 2007)
In Vietnam, the demand for learning English at the primary level has increased significantly among society, parents, and students In response to this demand, the Ministry of Education and Training (MOET) issued Decision No.50/2003 QĐ- BGD&ĐT on 30 October 2003, on the introduction of an optional English language programme at the primary level, in place of the pilot programme of 1996 According to the Decision, from 2003 to present, pupils in primary schools are required to learn a foreign language as an elective subject from Grade 3 to Grade 5 for two 40-minute periods per week, where there are adequate teaching conditions as well as demand from pupils and parents From 2006, it will become compulsory with four 40-minute periods per week
The MOET has implemented an English curriculum in primary schools The program focuses on the development of the four skills, with an emphasis on speaking and listening first The curriculum imposes strict restrictions on the number of words to be learned in each grade so that pupils can enjoy their English class without feeling the “burden” of the subject The specific numbers of words to be learnt are 120–140 words (for Grade 3); 140–160 words (for Grade 4); and 180–200 words (for Grade
5) There are two achievement tests, which include the first term test and the final test The tests include all four skills, and they are given at the end of the first and second semesters The percentage of each skill in each grade is totally different Table
1 shows the weightage distribution of the four skills in each grade
Table 1.1: Weightage distribution of four skills
At primary schools, the English achievement test has been settled as one of the most critical factors that can be used as a method of evaluation to determine whether or not pupils have met the learning target In other words, it is important to access the English achievement test because it will become the learning outcome for determining a student’s English proficiency
Because teachers’ perceptions and practices have a meaningful effect on their assessment and evaluation procedures (Breen et al, 1997; Arkoudis & O’Loughlin, 2004; Davison, 2004; Edelenbos & Kubanek-German, 2004) This research focuses on the teacher as the agent of the testing process (Rea-Dickins, 2004, p 251) Therefore, the English teachers teaching at the primary level in Thai Nguyen city will be participants in this study
Based on the reasons explained above and from the problems given, the researcher has been inspired and motivated to conduct the research on the teachers’ perceptions and practices with the title “An exploration into teachers’ perceptions and practices in designing English achievement tests at primary schools in Thai Nguyen city” with the belief that they will comprehend the reality of teacher-made English achievement tests and provide them with the required teacher development programs to help them improve their proficiency in teaching and testing It might be assumed that, as a result of this procedure, teachers will have the chance to think critically on why and what they do when testing their students.
Objectives and research questions of the study
This study aims to investigate teachers’ perceptions and practices in designing achievement tests Moreover, the present study also focuses on the differences between teachers’ perceptions and their practices in order to see to what extent the teachers have difficulties in designing English achievement tests
This study will address the following research questions:
1 What are teachers’ perceptions in designing achievement tests?
2 What are teachers’ actual practices in designing achievement tests?
3 What are the differences between teachers’ perceptions and practices in designing achievement tests?
Scope of the study
Teachers are the subjects of this study The object of this study is teachers’ perceptions and practices in teacher-made English achievement tests The research was conducted at primary schools in Thai Nguyen city in the second semester of the school year 2021- 2022 The study’s discussion focused on the teachers’ perceptions and practices in designing the teacher-made English achievement test for 5th grade students at primary schools in Thai Nguyen city.
Significance of the study
The purpose of this study is to explore the teachers’ perceptions at primary schools in designing English achievement tests Besides, it is necessary to see whether there are differences between their perceptions and their practices For better teaching, it is important to learn about how to design a good test because it can help teachers who have difficulties in preparing for English achievement tests And the findings of this study will be expected to provide readers with a description of an analysis of teachers’ perception and practice toward the English achievement test It can also be used as input for readers, particularly English teachers, headmasters, and other educators involved in or responsibility for developing high-quality tests In other words, it is beneficial for everyone to understand the characteristics of a good test as well as for researchers to use them as a basis for future research.
Organization of the study
The study is divided into five chapters as follows:
Chapter I: Introduction, which provides a broad view of the whole research by presenting the current problem, mentioning the aims and objectives of the paper, and indicating the significance as well as the scope of the study
Chapter II: Literature Review, which is supposed to provide a theoretical background on the issue of English language testing, discuss the key concepts, identify the research gap and review the related studies in history both in Vietnam and in the world
Chapter III: Methodology, which informs readers of the participants, the chosen instruments, procedures of data collection and analysis
Chapter IV: Findings and Discussion, which presents the description, and interpretation of the collected data in light of research questions
Chapter V: Conclusion and Recommendations, which summarizes the key points in the study, provides implications of the study, discusses limitations, and gives suggestions for further research.
LITERATURE REVIEW
Key concepts
In fact, there are many methods for assessing the teaching-learning process One method is to conduct a test Test is a way to measure the knowledgeability of a person or student According to Brown, test is a method of measuring a person’s abilities, knowledge or performance In line with Brown’s opinion, Sadar (2020) said that a test may be a tool, a question, a set of questions, or an examination that is used to measure a specific feature of an individual or a group of individuals Penny (1996) said that “Tests are used as a means to motivate students to learn or review specific material” (p 34) It means test is one motivation for students to learn or review material in their school Furthermore, Fernandes (1984) states that a test as a systematic procedure for surveying a person’s behavior and explaining it with the aid of a numerical scale or a category system
In addition, according to Linn and Gronlund (1995), “a test is a particular type of assessment that typically consists of a set of questions administered during a fixed period of time under reasonably comparable conditions for all students” (p.5) Globally, the test consists of a series of questions that must be answered correctly, with the purpose of determining an individual's ability to understand everyday learning
Testing is fundamental in any language program According to Brown’s
(1995) model of evaluating the elements of the language curriculum, since from the placement of students in the program through their graduation, all kinds of program- related decisions are made using the tests as a basis
Based on the aforementioned definitions, the researcher concludes that a test is a specific sort of evaluation used to reinforce learning and motivate students by assigning a task or a group of activities The teacher uses the test to not only evaluate and encourage pupils but also to enhance the lesson in the teaching and learning process To make an accurate decision, the teacher requires reliable data and the ability to gather data, which necessitates the use of a good instrument
2.1.2 Relationship of teaching, learning and testing
Testing is an element of the curriculum The relationship between tests, teachers, and students is so complex that how testing is done gains great importance
In some institutions each teacher is expected to write their own tests In some others, teachers working at the same level prepare the tests for their groups together Yet in some others, teachers do not prepare tests at all
Teaching, learning and testing become involved and related in a learning process like a cycle First, a teacher must teach the students a specific subject or matter, whether it is a specific grammatical structure, vocabulary, or even a story Then, the teacher must ensure that the learners completely understand the material that the teacher is providing At this stage, the four skills of reading, writing, speaking, and listening must be practiced by the students, and it is the teacher's responsibility to select the tasks that will be most useful to the students’ learning Finally, teachers should make tests Students should take the assessment provided by the teacher, and the suggestions that were made throughout the course, in order to apply them when the testing comes (Shohamy, 1993)
After this final stage, the cycle begins again, and as teachers, they must teach again, then assess, and finally evaluate However, at each stage, the teacher should consider some of the most important points, such as the ones mentioned above, in order to improve, simplify, and tailor the learning process to the needs of the learners.
In brief, testing has a direct impact on teaching because teachers utilize test results to modify their course objectives, make better use of the materials and activities in class, and, if they are the testers, design better tests (David, 1990) However, despite its importance, most teachers have little idea of what makes a good test or what qualities a test must possess (Basanta, 1995)
According to the previous explanation, one of the characteristics of a good test is validity The issue of validity is closely related to how good the test is, and this is acknowledged as a key notion in language testing and assessment (McNamara, 2001) Test validity is the most important issue to consider when judging foreign language testing The extent to which a test measures what it is supposed to assess is known as validity (William & Stephen, 1993) A test has validity if it accurately measures what it is designed to measure
In testing and assessment, validity has typically been defined as determining whether a test accurately measures what it is intended to measure or establishing the appropriateness of a particular test or any of its component parts as a measure of what it is intended to measure (Henning, 1987) The validity perspective is that when we construct a test, we have the objective to measure something, that the “something” is
“real”, and that validity investigation is concerned with determining whether a test
“actually does measure what is intended” (Glenn & Fred, 2007)
Based on the explanation above, it can be concluded that the tests can be considered as valid if they measure accurately and appropriately what they are intended to measure
2.1.4 Types and purposes of tests
As Brown (1995) points out, language tests are normally used to help in decision making, and they can be used for a variety of decisions In fact, we often refer to tests by the type of decision they are used to make These test and decision types can be divided into two major categories: those that are closely related to a teaching or learning curriculum and those that are not The writer use this distinction because curriculum-related tests all have a specific domain-the curriculum-to which we can refer when planning and writing these tests In contrast, when a test is not based on a particular curriculum, we have the burden or freedom (depending on one’s point of view) of deciding what specifically it should be based on
These types of tests are summarized in the table below Brief consideration, of course, will show that many tests are used for more than one purpose; the researcher will refer to several common types of overlap in the following discussion in Table 2 below
Table 2.1: Test Purposes and Types
Placement test Used to decide at which level in the language program a student should study Diagnostic test Used to identify learners’areas of strength and weakness
Used to assess how well students are doing in terms of mastering course content and meeting course objectives
Summative test Used to show the standard that the students have now reached in relation to other students at the same stage
The first type of curriculum-related test is placement tests A placement test is a related type of test that is used to determine which level of the language program a student should study at The student is then “placed” at that level, as the definition indicates In many cases, a single test may be used for both purposes: deciding whether a student’s language ability is adequate for even the lowest level of the program (admission decisions), and determining which level is most appropriate for them if they pass that barrier (placement decisions)
Diagnostic tests are used to identify a student’s strengths and weaknesses Diagnostic information is obtained from placement tests at times, but diagnostic tests are sometimes administered separately after students have been placed in the appropriate levels Diagnostic tests are also used by some language programs to ensure that students are placed correctly This can be a good idea, especially if a program isn’t sure about its placement procedures, but it’s arguable whether this has a diagnostic purpose (Bachman, 1991a, Brown, 1996, Heaton, 1988, Hughes, 1989)
Progress tests evaluate how well students are understanding course content and meeting course objectives This is done with the expectation that learning is still ongoing- that students are not expected to have mastered the content yet However, many progress decisions in the classroom are made informally, in the midst of teaching, rather than through testing (Leung & Mohan, 2004)
METHODOLOGY
Participants
The participants of the study were 18 teachers who are currently working at primary schools in Thai Nguyen city Among these teachers, there were two males and sixteen females Due to this distribution, gender was not examined as a variable Their experience as teachers’ ranges between one and more than 10 years
Three of these volunteers have Master’s degrees in English language; they are studying for a Master’s degree in English language at the School of Foreign Languages, Thai Nguyen University Furthermore, all of the teachers had enough experience and passion in English language teaching to be fully qualified take part in this study.
Data collection instruments
This section describes the tools used in the study for data collection According to Horwitz (1990), various data collection tools can be used to collect data such as questionnaires, interviews or observations
This study used both quantitative and qualitative methods Quantitative methods are those that focus on numbers and frequencies rather than on meaning and experience Quantitative methods (for example: questionnaires and experiments) provide information which are easy to analyse statistically and fairly reliable Qualitative methods are ways of collecting data which are concerned with describing meaning rather than with drawing statistical inferences (interview, observation) (Bryman, 1992)
Questionnaire surveys and interviews were used to gather the teachers’ perceptions and practices in designing English achievement tests By doing that, the researcher could get the descriptive data which are more reliable to answer the questions of the research
According to Creswell (2005), a questionnaire is a list of questions that are used in a survey and are completed by participants in the study before being returned to the researcher The participant responds to questions and provides basic demographic or personal information The questionnaire was chosen firstly for this study because it can reach a large number of people in a very short time Nunan (1992) noted that it is not only the quickest way of collecting information but also self- managed and can be given to large groups of subjects exactly at the same time Furthermore, questionnaires are an effective technique for collecting quantitative information and can be used to gather declarative data, but not performance data It is also useful to understand who, when, and where the various languages are spoken, as well as attitudes toward them Moreover, the Likert Scale was used for the second part of the questionnaire because it is regarded as a suitable and commonly used scale for eliciting the level of agreement or disagreement with certain expressions of opinion or attitude (Henning, 1987) The purpose of the questionnaire in this study is to seek information about teachers’ perceptions on the need of designing English achievement test and what the teacher thought about the representation of language skills and knowledge on their tests More specifically, the questionnaire consists of 3 parts:
Part 1 has 8 questions which aim to ask for the teachers’ background information such as full name, gender, age, academic qualification, years of teaching English, English skills that they teach at the school, whether they conduct the test by themselves or from outside and training course attendance on language testing
Part 2 has 16 questions which utilize closed items in which teachers could express their perceptions about the English achievement tests they designed in their classroom These questions are mostly based on a theoretical argument about the guidelines of English achievement tests
Part 3 also has 16 questions which used closed items in which teachers can decide what steps they actually took in their classroom In other words, this part examined the teachers’ practices in designing English achievement tests
Parts 2 and 3 of the questionnaires contain 32 items; these two parts contain essentially the same EAT procedural concerns, but from different viewpoints That is, while Part 2 requires teachers to respond to their own EAT working theories, Part
3 allows them to consider how well they place their perceptions into practice when conducting their tests Additionally, 32 propositions are grouped into four sub-areas based on procedure of testing principles: planning, implementation, monitoring, and recording and dissemination These are also linked to a variety of core EAT difficulties, such as validity and reliability
The results of the questionnaire provided a basis the construction of interview questions for the second investigation of the study Therefore, the interview and the questionnaire are reciprocal to each other
After administering the questionnaire, the data obtained were coded and run through SPSS to check the reliability of the items in the questionnaire The most popular statistical index that is used to measure the internal consistency of the questionnaire items is the alpha value, which was developed by Cronbach (1951) Cronbach’s alpha value satisfying 0.7 indicates an acceptable level of reliability, and 0.8 or above is a good level It is recommended that the benchmark Cronbach’s alpha value should be at least 0.7 Cronbach’s alpha value of satisfying 0.7 shows acceptable reliability If the Cronbach’s alpha is below 0.7, the evidence proves that the items do not have much in common and are unable to measure the same construct Table 3.1 presents the rules for alpha values suggested by George and Mallery (2003)
Table 3.1 Rules of alpha value
Values Levels α ≥ 90 Excellent α ≥ 80 Good α ≥ 70 Acceptable α ≥ 60 Questionable α ≥ 50 Poor α ≤ 5.0 Unacceptable Although it is ideal for all test items to measure the same latent variable, Cronbach Alpha can report high values in many cases even when the set of items examines different, independent variables (Cortina, 1993; Cronbach, 1951; Green et al., 1977; Revelle, 1979; Schmitt, 1996; Zinbarg et al, 2006) In order to increase statistical power, Iacobucci and Duhachek (2003) suggest computing standard errors and judging Cronbach Alpha together with a confidence interval After collecting and analyzing hundreds of articles and thousands of coefficients, Peterson (1994) described a typical level of 0.77, which signifies an acceptable average for all studies
Based on the interpretation of the alpha value, 32 question items were computed to investigate the reliability of the questionnaire Table 4.2 shows the alpha values of each questionnaire item in the present study
Table 3.2: Alpha values of the questionnaires
Table 3.2 indicates that the alpha values for all question items in the questionnaire are above 0.80, confirming the good level of the questionnaire It can be inferred that the items in the questionnaire are sufficiently reliable to conduct data for the present study
A semi-structured interview is a meeting in which the interviewer does not strictly follow a formalized list of questions Instead of a straight question and response approach, they will offer more open-ended questions, allowing for a structured interview format Both the interviewer and the candidate can ask questions, allowing for a detailed discussion of all relevant topics Because of the conversational tone, the candidate may feel more at ease discussing approaches and experiences that will highlight the qualities that make them a good fit for the position
Pilot study
After finishing the first draft of the questionnaire and receiving permission from the School of Foreign Languages, Thai Nguyen University to do the research, the researcher first administered a pilot study to check for any difficulties in understanding the items of the questionnaire in December 2021 with 02 teachers at primary schools in Thai Nguyen city In this section, the content validity questionnaire was piloted in order to test the applicability of the first version of the instruments for investigating the content validity of the EAT, make necessary modifications, refine the items in the questionnaire for the purpose of the main study and provide hands on experience to carry out the main study Firstly, the appointments with 02 English language teachers were made at the meeting room school Secondly, the instructions of the questionnaire were read out to make sure that they feel free to participate in the survey and ensure that the teachers would not have any difficulties with the items Thirdly, the teachers were asked to fill out their background information questionnaire and then they were asked to circle only one response for Question 1 to Question 32 The time for the teachers to complete the questionnaire was noted down Finally, the responses were checked
The survey showed that the instructions for the questionnaire were clear, so the teachers easily understood and were able to follow the instructions that were read to them from the instruction sheet and they felt free to fill in the questionnaire Furthermore, all teachers did not have any problems in filling out the background information questionnaire It took five minutes to fill out eight questions related to their personal information Then the second part lasted fifteen minutes for the teachers to answer 32 questions The teachers held the same view that items in the questionnaire reflected the content validity of the EAT and were easy to understand
In other words, it can be concluded that pilot participants did not have any problems with following instructions, completing the items within the time allocated and filling out the data collection instruments designed for questionnaires Thus, no modifications were made to the instruments and time was allocated for completing the items in the questionnaire From these findings, it can be concluded that the instruments used in the pilot study can be completely used in the main study.
Procedures of data collection
The first investigation collected data from January 2022 to March 2022, during the second semester at the primary schools
The research was carried out by series of investigations: firstly, a questionnaire survey was conducted to assess teachers’ perceptions and practices in designing English achievement tests; secondly, some volunteer teachers were interviewed to find out why they do what they do in relation to the issues identified in the first investigation The findings were then analyzed in light of the relevant materials developed and used for test activities
The data collection for the second investigation took place from mid-March
2022 to the beginning of May 2022
To conduct the research and to verify the research question, the following steps were taken:
Step 1: Identify and clarify research problems At this step, the author identified the research topic and evaluated it based on the problem to be solved by the enterprise
Step 2: Research the relevant theories and evaluation models after having identified the research problem
Step 3: Set up research models: After considering the relevant theories, research models have been used by other authors for different studies The researcher performs the research model selection based on previous theories At the same time, the author establishes a research model to suit specific research conditions
Step 4: Create questionnaires and interview questions for the research: The author sets up a questionnaire and interview questions to collect experimental data for the study
Step 5: Investigate to collect research data: After the questionnaire is corrected and completed, an interview will be conducted to collect data for further analysis purposes
Step 6: Analyzing the data: the data collected after being cleaned will be analyzed
Step 7: Present the research results: The research data analyzed will present the main results, compare with some similar studies to get accurate conclusions about the research results
Step 8: Conclude, recommend and complete the thesis: This is the last step of this study From the research results, the author will present the main conclusions of the study.
Data Analysis
Data analysis is the process used to present data and information collected from questionnaires and interviews In this study, some useful tools were used to calculate data from questionnaires and interviews, such as Microsoft Excel and SPSS Moreover, the results were illustrated in both tabular and graphical form
3.5.2 Procedure of analyzing the data
Stage 1: Check the questionnaires carefully to make sure that the respondents have answered all the questions The supplied answer must be understandable and valuable
Stage 2: Count and calculate data to compile statistics
Stage 3: Analyze the received data and present it in the form of tables and figures for better exemplification
Stage 1: Gather the data from interviews to collect more detailed information Stage 2: Analyze and discuss the data.
RESULTS AND FINDINGS
Results on Questionnaire
Research question 1: What are teachers’ perceptions in designing EAT?
From questions 1 to 8, the teachers’ background information is sought out, including their full name, gender, age, academic qualification, year of teaching English, English skills taught at the school, whether they conduct the test internally or externally, and attendance at language testing training courses
Figure 4.1 represents the information about the English teaching experience of teachers teaching at primary schools in Thai Nguyen city recently
Figure 4.1 Teachers’ English teaching experience
Years of teaching less than 5 years5-10 years11-15 yearsMore than 15 years
From Figure 4.1, it can be observed that half of the teachers have been teaching English at primary schools for 5 to 10 years 22% of the teachers have more than 15 years’ experience in teaching English, and 17% of them have worked for eleven to fifteen years This shows that English teaching has become entrenched in the primary education system, despite the fact that English is considered as a compulsory subject only has more than ten-years of history
However, two teachers (11% of the total) have fewer than five years of experience teaching English This suggests that many teachers are still interested in teaching English due to the fact that it needs a higher level of language skills
At primary schools, an English teacher teaches four skills: listening, speaking, reading, and writing Speaking and listening are emphasized in the second and third grades, while reading and writing are incorporated into the higher grades as students become more advanced Developmentally appropriate learning activities are used at each step to keep the focus on the learner’s autonomy and problem-solving abilities, which are the foundation for communicative competence It means that teachers have encountered many problems when they teach four skills
Figure 4.2 shows the different kinds of test that teacher frequently use in their classroom
As indicated in Figure 4.2, teachers often use progress tests and achievement tests All the teachers use achievement tests in the school years (100%) It means that achievement is the main tool used at primary schools Besides, there are two teachers (13%) who use both achievement tests and progress tests It can be easily understandable because they use achievement tests for the first term test and final tests; progress tests for daily checks
Figure 4.3 presents the frequency of tests that teacher used in the language classroom
Figure 4 3 Frequency of test carried out by teachers
Figure 4.3 indicates that more than 66% of teachers conducted the test twice According to current Ministry of Education and Training regulations, there are two periodicals exams at the primary level during the school year: at the end of semester
1 and at the end of semester 2 Additionally, 33% of teachers conducted the tests four or more than four times It appears that test practice has become a part of their regulary activity In contrast, there is no one to carry the test once and three times
4.1.2.5 Attendance the training course on language testing
Figure 4.4 presents the teachers’ attendance to the training course on language testing
Once Twice Three times Four times More than four times 0%
Figure 4 4 Attendance to the training course on language testing
The result in Figure 4.4 shows that a large number of teachers (83%) took part in the training course on language testing Only a small percentage of teachers (17%) who have only been teaching English for two or three years have not taken any courses on language testing This indicates that a majority of teachers were provided with good testing knowledge when they designed the English test
Figure 4.5 shows the sources of the tests the teacher used when they conducted the English tests
Attendance to the training course on language testing
Used both self- developed test and test from outside source
Used both self-developed test and test from outside source
From Figure 4.5, it is clearly seen that the majority of teachers (76.5%) created their own tests The rest (23.5%) used both self-developed tests and tests from outside sources This demonstrates that teachers might have difficulties finding ready-made tests or test materials, or that they consider the appropriateness of current tests or test materials for their achievement tests at issue
4.1.3 Teachers’ perceptions in designing EAT
This section examines the perceptions that teachers use to drive their tests of students’ achievement in their classes
In order to establish teachers’ perceptions in designing the achievement tests, descriptive statistics analysis was conducted by computing the mean scores for each item in four stages: planning, implementation, monitoring, and recording and dissemination in the teachers’ questionnaire from question 1 to question 16 Table 4.3 presents the interpretation of the mean scores:
Table 4.1 The interpretation of the mean scores
Note: VH=Very high, H=High, M=Moderate, L=Low, VL=Very Low
The criteria for the mean scores adopted from Kucuk (2007) are shown in Table 4.1 Five Likert-scale criteria were used to assess the degree in which the respondents agreed with their principle in designing achievement tests More specifically, the strongest agreement is between 4.5 and 5.0, followed by 3.5 to 4.4, and the undecided option is between 2.5 and 3.4 Finally, the strangles disagreement ranges from 1.0 to 1.4, while the disagreement ranges from 1.5 to 2.4
In a nutshell, mean scores in Likert-scale criteria are used to assess a participant’s attitude by determining to what extent they agree or disagree with a specific question or statement
Table 4.2 reveals some teachers’ perceptions that they use as a reference when preparing test of students’ achievement in the classroom
Table 4.2:Teachers’ perceptions in the planning stage
Items Propositions Mean SD Degree
Teachers should first identify the purpose of the test and tests must be related to what students do in real class time when they design the test
Teachers should consider the attainment targets which the curriculum requests when they design the test
Teachers should use test specifications when they carry out the test
The appropriateness of test (tasks) should be checked by calling for peer comment or with reference to published guidelines (if these are available)
Teachers should give the students advance notice, so that the students will be able to prepare for the test
Note: VH=Very high, H=High, M=Moderate, L=Low, VL=Very Low
Propositions 1 and 2 receive the most positive comments in the first stage (Table 4.2), with the highest mean score (M = 4.33, SD =.485) Put it simpler, teachers agreed that they should first identify the purpose of the test, which is linked to what students do in practical class time when they design the test, and then they should identify the achievement targets that the curriculum requires when designing the test Meanwhile, the least positive comment is for proposition 4 (M = 3.28, SD = 1.114), in which they consider that the appropriateness of the test should be reviewed by calling for peer comment or with reference to published guidelines The rest ranges from 4.22 to 4.28, indicating the fact that teachers should use test specifications when they carry out the test and that teachers should give the students advance notice, so that they will be able to prepare for the test
The overall mean score is accounted for at 4.188, demonstrating that all of the propositions submitted in the questionnaire were judged by the teachers to be part of their own evaluation principles In descending order of means, the teachers who replied usually regarded the propositions given at this stage as part of their own test principles Propositions 1, 2, 3, and 5 appear to be the principles that most teachers believe in and are most likely to be implemented in achievement tests Thus, these findings suggest that teachers recognize the importance of developing a test strategy and understanding how to implement it
Table 4.3 presents teachers’ perceptions in the implementation stage It can be seen that the overall mean score of the implementation stage is (M = 4.32, SD =.505), showing that all of the propositions presented in the questionnaire were perceived by the teachers to be part of their own test principles
Table 4.3: Teachers’ perceptions in the implementation stage
Items Propositions Mean SD Degree
Teachers should inform the students of the reasons why they are being tested
Teachers should explicitly instruct the students how to do the test (tasks)
Teachers should give students immediate feedback after they complete each test (task)
Test (tasks) processes should be completed within a manageable time considering the given context
Note: VH=Very high, H=High, M=Moderate, L=Low, VL=Very Low
Table 4.3 depicts proposition 8 ranks first with the highest mean score (M 4.39, SD = 0.502), followed by P 7 with the mean score of 4.33 (M=4.33, SD=.485) while the lowest mean score (M= 4.28, SD= 575) is accounted for the propositions
CONCLUSION AND RECOMMENDATION
Overall discussion
5.1.1 Research question 1: What are teachers’ perceptions in designing EAT?
When planning English achievement tests, the teachers seem to have a strong understanding of what is valid This leads to a variety of problems that mirror features of their testing context Concerning the purposes of tests (Propositions 1 and 2), it appears that teachers understand the importance of planning for achievement tests They first organize their test by comparing the students’ achievement tests to the curriculum’s objectives, and then reflecting on the students’ learning experiences These findings suggest that the teachers’ perceptions of the test are consistent with one of the achievement test’s characteristics The findings are in line with Arkoudis and O’Loughlin (2004), who claimed that teachers should deeply understand the process of testing and assessment of learning for academic purposes It appeared that the teachers were primarily concerned with testing for “educational purposes”
Furthermore, the teachers truly believed that they need to apply their judgments based on a certain level of professional expertise because they were sometimes forced to choose between the published technical specifications and their own professional judgment (Davison, 2004, p 316) This also demonstrates that, while the national curriculum has a significant influence on what happens in the classroom, it is nonetheless localized to some extent through the teachers’ own perception Thus, teachers in the middle of the five-staged cline of teachers could be classed as “assessors” (Davison, 2004, p 325) This indicates that they intend to reconcile the required criteria with contextual considerations as principled yet pragmatic experts Sharing the same view, several researchers (Davison, 2004, Shim,
2009) highly agree that teachers should assume a role as assessors who think critically, creatively, and logically in order to assess a student’s performance in order to give the teacher an idea of how students learn, their attitudes, what skills they have, what skills they are working on, and how they are progressing
As indicated in the preceding section (Section 2.1.3, Chapter II), for a test to be called “good”, it must be accomplished through appropriate procedures; that is, it cannot be anticipated to come up with a real result or feedback on what the students know or can achieve unless the test is done in a rational way (Proposition 5) Teachers’ opinions are typically in accordance with the concepts stated in this study during the implementation stages; they have a good understanding of what a legitimate method and ethical evaluation is
Teachers’ perceptions are typically in line with the principles stated in this study during the implementation stages; they have a good understanding of what a valid procedure is This raises a number of issues which should be addressed Teachers considered that students should understand how to accomplish the tasks and attempted to finish the test within the time given They appeared to be acting as an
“interpreter” (Davison, 2004, p 325) who followed the guide’s suggestions The results for specific guidance on how to complete the test tasks (Proposition 7) and task completion within an expected time (Proposition 9) reveal that the teachers’ test principles are compatible with one of the features of EAT, namely that EAT is impacted by both the national and school curriculum
During the monitoring stages, teachers have a firm understanding of how to use test findings and how to acquire consistent marks or information on students’ achievement or performance However, there are a number of difficulties that must be addressed First, it shows that the teachers were confident in their use of test results They stated that they utilized test findings largely to reflect on what they did in the classroom and then to improve their teaching (Proposition 12) As a consequence of these findings, the teachers’ perceptions were found to be mostly similar to the argument that testing is not an isolated event but is linked to a full set of variables that interact in the educational process (Shohamy 1993; Shohamy et al
1997) Hughes (1993) defines this process as material development, syllabus design, changes in teaching methodologies, and learning and test- taking strategies Teachers’ understanding of the washback effect was restricted in this study because their responses indicated that they used the assessment findings primarily to revise their teaching approaches rather than to analyze other areas such as syllabus design and textbooks used This might be due to the fact that they were given standardized syllabi and the MOET created most of their materials This may have reduced teachers’ understanding of the importance of challenging or revising them Second, the teachers were concerned about the negative usage of test results They thought that the test results could be used negatively in the classroom in very few, if any, instances They were, however, unaware of the use of test results by others involved with EAT This lack of awareness of the use and misuse of EAT results could be solved by improving their professional competence
In recording and dissemination stage, the teachers believed that their school should develop their own report systems and that they should participate the development process (Propositions 14 and 15) Proposition 16 relates to the power relationships of tests According to the findings of the reporting of students’ progress and achievement, teachers were moderately worried about whether or not they should report their students’ educational results to the local educational authority or the government
In conclusion, the study’s findings demonstrated that the majority of teachers have strong perceptions about EAT and have a good understanding of testing principles; therefore, they conducted their tests using an appropriate process while keeping in mind the English teaching and testing context
5.1.2 Research question 2: What are teachers’ practices in designing EAT?
In the planning stage, Propositions 17, 18, and 21 were discovered to be consistent with what the teachers believed This shows that the teachers generally followed their concepts; specifically, they scheduled tests and created test tasks that reflected their understandings The test tasks were compatible with what was provided in the textbooks and accurately reflected the objectives of the curriculum However, there were two situations where they did not usually act in line with their perceptions Using test specifications when carrying out the test (Proposition 19) and checking the appropriateness of the tasks with other teachers (Proposition 20) did not seem to be usual parts of testing practice
In the implementation stage, the teachers did perform in most items in line with their perceptions; compared to the other stages, there is greater agreement between teachers’ perceptions and their practices To some extent, all propositions were shown to be applicable to what teachers actually did in implementing the test This stated that the teachers tended to follow their concepts, that is, they carried out their tests as planned within the semester’s curriculum The instructions were simple for the pupils to understand It also indicated that the test tasks could be completed in the allotted time, either during class or at a specific time However, when compared to the other propositions, providing immediate feedback (Proposition 24) was shown to be less of a feature of their testing process at this stage
Propositions 26, 27, and 28 could be considered as more closely reflecting the preferred principles of the teachers during the monitoring stage; these are more likely to be carried out through to the EAT The teachers considered constructing a making system as a part of the whole testing process, which is an essential step in designing EAT Furthermore, developing criteria that are related to the test’s objectives and using the test results to revise their teaching are critical steps in the real world
However, sharing the students’ test results with other colleagues seemed not to be done by the teachers, and therefore did not often occur in their testing practice
In most cases, the teachers did not actually act in accordance with their perceptions; that is, their practice did not reflect their principles Proposition 31 was found to be applicable to what the teachers did on their students’ tests This demonstrates that the teachers agreed that they should be involved in the process However, Propositions 30 and 32 were hardly ever implemented as part of the teachers’ testing practice In fact, it is not easy for teachers or even administrators to develop their own reporting system for achievement and a formal review of students’ progress and achievement It requires the leadership, guidance, and close collaboration of teachers, schools, and authorities
In brief, this study suggests that teachers did not always put some of the theories that they believe in practice when carrying out their tests
5.1.3 Research question 3: What are the differences between teachers’ perceptions and practices in designing EAT?
As mentioned in Sections 4.1.3 and 4.1.4, the teachers recognized the significance of the planning stage However, the teachers’ real use of test specifications (proposition 3) suggests a number of considerations which need to be discussed If specifications are not given to the test validator, he or she will be unable to learn about the testers’ criteria for sampling what has been taught in class, their testing objectives, and performance expectations from the students (Alderson et al, 1995; Hughes, 1989) Unfortunately, most teachers do not have such clearly defined objectives to refer to, and so tend to test only what is simplest to test (Davies, 1990; Hills, 1976; Hughes, 1989) This is in line with one of the arguments put forward by Brown (1996), that the biggest problem for classroom tests is that it is usually not possible to find carefully prepared test specifications and well-defined course objectives to compare the tests against
Implications of the study
In this section, the researcher identifies the significant concerns and their implications for English teaching and testing, which require more investigation and, ideally, will lead to improvements in classroom practice Then the researcher concludes with a few recommendations
The study discovered that teachers understood what a good test was, that they typically had their own testing principles, and that they put them into actual practice Clearly, they were not passive recipients who just followed what was handed to them from outside, but rather active practitioners who conducted their tests in a self- directed manner They attempted to analyze all conceivable elements that should be considered when analyzing their specific pupils, such as the curriculum, the schooling system, the function of the administrator, and even the interaction between classroom teachers and English teachers in the school Therefore, there is a significant need for extensive research of teachers’ perceptions and practices, which might cover the following themes: Teachers “diagnostic competency” (Edelenbos & Kubanek- German, 2004) and real diagnostic actions in assessment activities
Additionally, there might be a need for extensive empirical study on the influence of formal test programs supplied by official and private institutions, as well as EAT developed by English teachers This is particularly crucial because it links to the test washback effect, which allows teachers to reflect on their teaching and testing and subsequently adjust what they have done They may also improve their grasp of test theory and principles, as well as their own test principles, by reflecting on the effects of tests created by other organizations as well as their own tests
Besides that, research is necessary on the role and impact of head teachers, supervisors, and managers who possess significant power and make judgments on the use of test materials from outside organizations such as the national testing agency, local educational authorities, and even publishing companies, as well as on test implementation by the school itself
Last but not least, there is a need for study on what it would take to change the teachers’ and other stakeholders’ perceptions toward language testing As a result, their school testing practices have evolved schools over time This is something that has to be addressed as a developmental concern
This study discovered two major difficulties in designing EAT at primary schools in Thai Nguyen City: firstly, the quantity of students in English classrooms; and secondly, the teachers’ work overload, which limited their ability to teach and test
However, these issues can not be easily resolved just by teachers or their schools, but they can be handled by bureaucratic and political efforts A persuasive justification must be made before society as a whole commit to increasing the number of English teachers and reducing or reorganizing teachers’ teaching loads These challenges might not be handled quickly Undoubtedly, local and national governments must develop a long-term strategy that involves what qualifications
English teachers need, how they are employed, and how current English teachers’ competencies can be improved
This research has also brought into question certain features of the existing system for providing English teaching in primary schools Although English has been taught for over ten years by English teachers at primary schools, it is essential to investigate the school system by which individuals can become English teachers
This is tied to the question of how English teaching should be regarded in connection to the whole primary school curriculum; if this is clearly defined, both schools and school administrators can work toward effective English teaching
Finally, this study discovered a significant mismatch between the functions of classroom teachers and English teachers, which relates to both English teaching and testing Obviously, there must be tight collaboration between English teachers and the classroom teachers; it is essential to stimulate the students’ English language acquisition This is because students’ daily school life and school achievement are greatly impacted by their classroom teachers Therefore, it could be a good idea to clarify the distinctions between the roles of English teachers and classroom teachers; this should be followed by an exchange of views regarding the English teaching and testing process A detailed action plan should be created to handle the problem in order to ensure that this actually happens The ultimate goal is to make sure that every effort is made to help students learn English, so both English teachers and regular classroom teachers contribute to this goal.
Limitations of the study
This study is of no exception of the fact that “nothing is perfect and limitations are inevitable.”
The first limitation of the study comes from the limited size of a minor thesis; the research was conducted only on a small sample size, on teachers Future studies may focus on students and administrators to increase the content validity It may also be a good idea to find out what the students and administrators think about the achievement tests because they are actually the ones who use these tests and are directly affected by the results and interpretations of the achievement tests
The second limitation caused by the limited time, so the content validity data were collected from questionnaire and interview only Another possible research may use observation, think-aloud protocols and stimulated recall to elicit more extensive data from the expert judgments Furthermore, with only 3 months, the study only focused on the content validity of the EAT; future studies may expand on face validity, concurrent validity, and predictive validity to increase the EAT’s validity and reliability
Due to the covid-19 pandemic, data collection was difficult, the researcher could not collect data directly, but must collected data online That is the third limitation of the study that still leaves the issue open for discussion.
Suggestions for further research
This researcher hopes that this study will be useful not only for those currently involved in English teaching and testing in schools, such as teachers, principals, and students, but also for other stakeholders interested in English language testing, such as students’ parents, supervisors, administrators, and policymakers Furthermore, this research might serve as a springboard for anyone interested in conducting more extensive research in this field
Despite its limitations, this study provided the researcher a great opportunity to learn more about the theory and principles of EAT, as well as about teachers’ practices and perceptions in designing EAT in English teaching and learning in Thai Nguyen city This could pave the way for future study as well as the researcher's ongoing participation in teacher development and English language testing programs.
Concluding remark
This study investigates teachers’ perceptions and practices in designing English achievement tests at primary schools in Thai Nguyen city Like other research studies on language testing and assessment, the present study also has some limitations However, despite the limitations, the findings from the study have shed light on important factors which relate to the effect of the process of test preparation and test construction Thus, it is further hoped that this study will be a contribution to the ongoing efforts to provide more validity evidence for English language tests
Abidin, S Z., & Bannur, F M (2015) A validation process of ESP Testing using
Weir's Socio Cognitive framework (2005) Science direct
Arikunto, S (2010) Dasar-dasar Evaluasi Pendidikan Jakarta: Bumi Aksara
Arkoudis, S., & O'Loughlin, K (2004) Tensions between validity and outcomes: teacher assessment of written work for recently arrived immigrant ESL students Language Testing, 284-304
Bachman, L F., & Palmer, A S (1996) Language Testing in Practice UK: Oxford
Basanta, C P (1995) Coming to Grips with Progress Testing: Some Guidelines for
Brown, H (2003) Language Assessment Principle and Classroom Practice New
Brown, H D (2004) Language Assessment: Principles and Classroom Practices
Brown, J D (1995) The elements of Language Curriculum Boston: Heinle& Heile
Brown, J D (1996) Testing in Language Programs New York: Prentice Hall
Brown, J D., & Hudson, T (1998) The alternatives in Language assessment
Bryman, A (1992) Quantity and Quality in social research London: New York
Childs, R A (1989) Constructing Classroom Achievement Tests Retrieved from
ERIC Digest: http://www.ed.gov/databases/ERIC Diuests/ed 315426.html
Cortina, J M (1993) What is coefficient alpha? An examination of theory and applications Journal of Applied Psychology, 78(1), 98-104
Creswell, J W (2005) Educational research: Planning, conducting, and evaluating quantitative and qualitative research Upper Saddle River
Cronbach, L J (1951) Coefficient alpha and the internal structure of tests
David, A (1990) Principles ofLanguage Testing Padstow: Cornwall: Basil
Davison, C (2004) The contradictory culture of teacher-based assessment: ESL teacher assessment practices in Australian and Hong Kong secondary schools Language Testing 21(3), 305-334
Edelenbos, P., & Kubanek-German, A (2004) Teacher assessment: the concept of diagnostic competence Language Testing, 21(3), 259-283
Fernandes (1984) Testing and Measurement Jakarta: Nasional Education
Field, J E (2013) "Cognitive validity" in Examining listening: Research and practice in assessing second language eds A Geranpayeh and L Taylor
George, D., & Mallery, P (2003) SPSS for Windows step by step: A simple guide and reference 11.0 update (4th ed) Boston, U.S: Allyn & Bacon
Glenn, F (2010) Practical Language Testing Britain: Hodder Education, An
Glenn, F., & Fred, D (2007) Language testing and assessment: An advanced resource book Routledge: London and New York,
Hawkey, R (2013) Consequential validity In A Geranpayed, & L Taylor, Studies in Language Testing: Examining Listening Cambridge University Press
Heaton, J (1988) Writing English Language Tests Hong Kong: Longman Group
Henning, G (1987) A guide to Language Testing: Development, Evaluation,
Research Boston: Heinle &Heinle Publishers
Horwitz, E K (1990) Teaching and learning in the language classroom Oxford,
United Kingdom: Oxford University Press
Hughes, A (1989) Testing for Language Teachers Cambridge: Cambridge
Iacobucci, D., & Duhacheck, A (2003) Advancing Alpha: Measuring Reliability
With Confidence Journal of Consumer Psychology, 13, 478-487
Jung, S K., & Norton, B (2002) language planning in Korea: The new elementary english program in J.W Tollefson (ed.) Language Policies in Education: Critical Issues, 245-265
Kopalle, P K., & Lehmann, D R (1997) Alpha inflation? The impact of eliminating scale items on Cronbach's alpha Organizational Behavior and
Kucuk, P (2007) The relationship between face validity, realiability and predictive validity ò University English Foreign Language preparatory school achievement tets Turkey: Kilkent University
Kuntasal, I (2001) Perceptions of teachers anf testers of achievement tests prepared by testers in the Department of Basic English at Middle East
Technical University (Unpublished Master's Thesis) Turkey: Bilkent
Leung, C., & Mohan, B (2004) Teacher formative assessment and talk in classroomcontexts: assessment as discourse and assessment of discourse
Lim, G S., & Khalifa, H (2013) "Criterion-related validity" in Examining
Listening: Research and practice in assessing second language listening, 35 eds A
Linn, R L (1995) Measurement and Asessment in Teaching, Seven Edition
Madsen, H (1983) Techniques in testing Oxford: Oxford University Press
McNamara, T (2001) Language assessment as social practice: challenges for research Language Testing, 18(4), 333-349
Norman, G E (1968) Constructing Achievement Tests London: London: Prentice
Nunan, D (1992) Research Methods in Language Learning Cambridge: CUP
Penny, U (1996) A course in language Teaching: Practice and Theory
Peterson, R A (1994) A Meta-analysis of Cronbach's Coefficient Alpha Journal of Consumer Research, 21(2), 381-391
Rea-Dickins, P (2004) Understanding teachers as agents of assessment Language
Rudman, H C (1989) Intergrating testing with teaching Eric Clearinghouse on
Tests Measurement and Evaluation (ED315432) Retrieved from ed.gov: www ed.gov/databases/ERIC_Digests/ed315432.html
Sadar, S (2020, 4 25) Test, Measurement, and Evaluation Retrieved from idoc.pup: https://idoc.pup/documents/dr-sanjit-sadar-test-measurement-and- evaluationpdf accessed on 25/4/2020
Shim, K N (2009) An investigation into teachers' perception of classroom-based assessment of English as a foreign language in Korean primary education
Shohamy, E (1993) The power of tests: the impact of language tests on teaching and learning NFLC Occasional Papers
Talor, C S (2013) Validity and Validation: Understanding statistics New York:
Taylor, L., & Geranpayed, A (2011) Assessing listening for academic purposes: defining and operationalizing the test construct J Engl Acad Purp, 89-110
Thu, D M (2020) Washback of an English achievement test on teachers'perceptions at Vietnamese University VNU Jounal of Foreign studies, 36(3), 178-194
Van, H V (2007) The current situation and issues of teaching of English in
Vietnam Teaching English to speakers of other languages in the internationalization of higher education in Vietnam, 3 (4), 1-12
Weir, C J (1995) Understanding & Developing Language Tests UK: Phoenix
Weir, C J (2005) Language testing and validation London: Palgrave Macmillan
Weir, C J (2005) Language Testing and Validation London: Palgrave Macmillan
William, W., & Stephen, G J (1993) Educational Measurement and Testing, second edition Boston: Allyn and Bacon
Xi, X (2008) Methods of test validation In E Shohamy & N.H Hornberger
(Eds.) Encyclopedia of language and education, 2nd edition, volume 7:
APPENDIX 1 : Teachers’ questionnaires APPENDIX 2 : Interview questions teachers
I am a postgraduate student at School of Foreign languages, Thai Nguyen University My M.A research focuses on the perceptions and practices of teachers in designing the achievement test at primary schools in Thai Nguyen city
I would like to invite you to kindly participate in this study by completing the enclosed questionnaire The survey should take about 10-15 minutes to complete Your participation in this study is voluntary and you are free to withdraw your participation from this study at any time There are no risks associated with participating in this study All responses in the survey will be recorded anonymously and used for the purpose of this research only
If you agree to participate in this study, please respond to the items in the questionnaire Your honest opinions are very crucial for this research
If you have any questions regarding the survey or the research in general, please feel free to contact Dinh Thi Lien at dinhlien44030@gmail.com
Thank you for your cooperation and time
Section 1 The first eight questions below will give demographic data to the researcher Please circle the appropriate answer for you
1 What is the name of the school where you teach?
2 What is you highest qualification?
3 How long have you been teaching English?
4 What English sub-skills are you in charge of teaching? (You can tick more than one option)
5 What kinds of test do you frequently use during the academic year? (You can tick more than one option)
6 How many times do you carry out these tests Please specify
7 Have you ever attended any training courses on language testing?
8 Do you use tests provided by outside sources or construct them for yourselves when you assess your students’ achievement?
I use tests constructed by institutes outside or publishing companies
I construct the tests for myself
PART 2-1: Teachers’perceptions in designing English achievement tests
This section aims to seek information about teachers’ perceptions on the need of designing English achievement test Please circle only ONE of the options given
Strongly disagree Disagree Neutral Agree Strongly agree
1 Teachers should first identify the purpose of the test and tests must be related to what students do in real class time when they design the test
2 Teachers should consider the attainment targets which the curriculum requests when they design the test
3 Teachers should use test specifications when they carry out the test SD D N A SA
4 The appropriateness of test (tasks) should be checked by calling for peer comment or with reference to published guidelines (if these are available)
5 Teachers should give the students advance notice, so that the students will be able to prepare for the test
Strongly disagree Disagree Neutral Agree Strongly agree
6.Teachers should inform the students of the reasons why they are being tested SD D N A SA
7 Teachers should explicitly instruct the students how to do the test (tasks) SD D N A SA
8 Teachers should give students immediate feedback after they complete each test (task) SD D N A SA
9 Test (tasks) processes should be completed within a manageable time considering the given context
Strongly disagree Disagree Neutral Agree Strongly agree
10 Teachers should construct a marking system as a part of the whole test process SD D N A SA
11 Marking criteria should be connected with the aims of the test and the learner’s characteristics in a given context
12 Teachers should use the results of test for revising their teaching SD D N A SA
13 The overall feedback should enable students to know how to improve their work and take their learning forward
Strongly disagree Disagree Neutral Agree Strongly agree
14 Schools should develop their own report system of students’ progress and achievement SD D N A SA
15 Teachers should be involved in the development of the report system at all levels SD D N A SA
16 A formal review of a student’s progress and achievement should be reported to the local education authority and the central government
PART 2-2: Teachers’ practices in designing English achievement tests
For each of the items please circle the score that best represents the degree of correspondence between your personal working theory and your actual practice That is, to what extent do you put your personal theory into your practice?
Before the test is designed
Never Hardly ever Sometimes Quite often Very often
17 I first identify the purpose of the test and the test that I desiged are always related to what students were taught in the classroom
18 I consider the standards or attainment targets which the curriculum requests 1 2 3 4 5
19 I use test specifications when I carry out the test 1 2 3 4 5
20 The appropriateness of test (tasks) are checked by calling for peer comment or with reference to published guidelines (if these are available)
21 I give the students advance notice, so that the students are able to prepare for the test 1 2 3 4 5
Never Sometimes Quite often Very often
22 I inform the students of the reasons why they are being tested 1 2 3 4 5
23 I explicitly instruct the students how to do the test (tasks) 1 2 3 4 5
24 I give students immediate feedback after they complete their test 1 2 3 4 5
25 Test (tasks) processes are completed within a manageable time considering the given context
Never Hardy ever Sometimes Quite often Very often
26 I construct a marking system as a part of the whole test process 1 2 3 4 5
27 Marking criteria should be connected with the aims of the test and the learner’s characteristics in a given context
28 I use the results of test for revising my teaching 1 2 3 4 5
29 The overall feedback should enable students to know how to improve their work and take their learning forward
Never Hardy ever Sometimes Quite often Very often
30 My schools develop their own report system of students’ progress and achievement 1 2 3 4 5
31 I am aware of their responsibilities for the output of their professional work 1 2 3 4 5
32 I am involved in the development of the report system at all levels 1 2 3 4 5
The analysis of the first questionnaire indicates that there is something of a mismatch between what teachers perceive and what they actually do with regard to EAT Please answer the following questions
1 Did you refer to test specification when constructing the test items?
2 Did you peer checked the appropriateness of test with your colleagues when planning the test?
3 Did you give immediate feedbacks after students complete their test?
4 Did you share the findings of the test with other teachers?
5 Did your school develop your own report system of students’ progress and achievement?
6 Did you/ your school make a formal review of a student’s progress and achievement that report to a local education authority and the central government? Thank you very much for your help!