Future research may make use of longitudinal designs to monitor learners' changes in English language ability over time. Studies over longer periods of time would shed
44
light on the course of language development and the successful outcomes of learning approaches.
In addition, in order to improve the generalizability of the results, future studies should strive to enlist a wider range of participant groups, such as those with varying age groups, educational backgrounds, and proficiency levels. It is beneficial to understand how people from different educational backgrounds engage with reading materials and overcome comprehension obstacles. Moreover, examining demographic elements like socio-economic status, cultural background, and language exposure can enhance our comprehension of the socio-cultural impacts on language acquisition and reading comprehension. Investigating how differences in socio- economic status, linguistic variety, and cultural practices influence learners' perspectives, motivations, and reading habits can help create equitable educational strategies and ensure fairness for candidates. This would provide a more thorough comprehension of the variables affecting the results of English language acquisition.
In addition, it is clear that more validation research is required to support the current research's conclusions and demonstrate the reliability of the VSTEP reading exam as a gauge of English competence. This could involve providing the sample test to larger, more representative groups of students and carrying out further research to evaluate the validity and reliability of the outcomes.
Furthermore, qualitative research might supplement quantitative analysis to offer a better understanding of learners' reading comprehension experiences, views, and methods. Qualitative research might investigate elements like linguistic attitudes, motivation, and sociocultural impacts on language acquisition.
By pursuing these recommendations for future further research, it is vital to build a more thorough understanding of English language assessment and learning in the Vietnamese setting, which will eventually guide the creation of more efficient language education methods and regulations.
45
REFERENCES
Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. Educational and psychological measurement, 40(4), 955-959.
Aini, Q., Rahardja, U., & Naufal, R. S. (2018). Penerapan Single Sign On dengan Google pada Website berbasis YII Framework. Sisfotenika, 8(1), 57-68.
Alderson, J., Clapham, C., & Wall, D. (1995). Language test construction and evaluation. Cambridge University Press.
Allen, M. J., & Yen, W. M. (2001). Introduction to measurement theory. Waveland Press.
American Educational Research Association, American Psychological Association, &
National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
Anastasi, A. & Urbina, S. (1997). Psychological testing (4th edition). Upper Saddle River, New Jerssy: Prentice Hall
Azwar, S. (2017). Reliabilitas dan Validitas (Edisi 4). Yogyakarta: Pustaka Pelajar.
Babbie, E. (2016). The practice of social research. Cengage Learning.
Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests (Vol. 1). Oxford University Press.
Bachman, L.F. (1990). Fundermental Considerations in Language Testing. Oxford University Press.
Bart, A. (2008). Enhancing content validity and internal consistency reliability in standardized English proficiency tests. Journal of Language Assessment, 10(2), 123-137.
Bart, A., Smith, J., & Johnson, R. (2009). Assessing test reliability: A systematic approach. Language Testing, 15(3), 211-226.
Berchtold, A. (2016). Test–retest: agreement or reliability?. Methodological Innovations, 9, 2059799116672875.
46
Biber, D., van Dijk, T. A., & Kintsch, W. (1986). Strategies of Discourse Comprehension. Language, 62(3), 664. https://doi.org/10.2307/415483
Bowling, A. (2005). Just one question: If one question works, why ask several?. Journal of Epidemiology & Community Health, 59(5), 342-345.
Brown, H. D. (1994). Principles of language learning and teaching. Englewood Cliff. NJ: Pentice.
Brown, H. D., & Lee, H. (2015). Teaching by principles: An interactive approach to language pedagogy (4th ed.). New York: Pearson Education.
Brown, J. D. (2018). Testing in language programs: A comprehensive guide to English language assessment. McGraw-Hill Education.
Cohen, L., Manion, L. & Morrison, K. (2000). Research Methods in Education.
Routledge/Falmer.
Cook, D. A., & Beckman, T. J. (2006). Current concepts in validity and reliability for psychometric instruments: Theory and application. The American Journal of Medicine, 119(2), 166.e7–166.e16.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests.
Psychometrika, 16(3), 297-334.
DeVellis, R. F. (2016). Scale Development: Theory and Applications (4th ed.). Los Angeles: Sage Publications.
DeVon, H. A., Block, M. E., Moyle‐Wright, P., Ernst, D. M., Hayden, S. J., Lazzara, D. J., ... & Kostas‐Polston, E. (2007). A psychometric toolbox for testing validity and reliability. Journal of Nursing scholarship, 39(2), 155-164.
Fielding, N. G., Lee, R. M., & Blank, G. (2008). The SAGE handbook of online research methods. Sage Publications.
Fraenkel, J., Wallen, N., & Hyun, H. (1993). How to Design and Evaluate Research in Education 10th ed. McGraw-Hill Education.
Frisbie, D. A. (1988). Reliability of scores from teacher‐made tests. Educational measurement: Issues and practice, 7(1), 25-35.
Fulcher, G., & Davidson, F. (2007). Language testing and assessment. London and New York: Routledge.
47
Haynes S. N., Richard D. C. S., Kubany E. S. (1995). Content validity in psychological assessment: A functional approach to concepts and methods. Psychological Assessment, 7, 238-247.
Hughes, A. (1989). Testing for language teachers. Cambridge: Cambridge University Press.
Imsa-ard, P. (2019). TOEIC reading section: Evaluation of the four cardinal criteria for testing a test. NIDA Journal of Language and Communication, 24(36), 91- 110.
Koller, I., Levenson, M. R., & Glück, J. (2017). What do you think you are measuring?
A mixed-methods procedure for assessing the content validity of test items and theory-based scaling. Frontiers in psychology, 8, 126.
McMillan, J. H., & Schumacher, S. (2001). Research in education: A conceptual introduction. Longman.
McNamara, D. S., & Kintsch, W. (1996). Learning from texts: Effects of prior knowledge and text coherence. Discourse processes, 22(3), 247-288.
Nguyen, T. (2017). Culturally relevant test content in Vietnamese standardized English assessments. International Journal of Language Education, 5(1), 45- 58.
Nguyen, T. H., Le, H. P., & Tran, N. T. (2018). Aligning the VSTEP test with the Vietnamese English language curriculum. Journal of Language Testing and Assessment, 12(2), 45-58.
Nunnally, J. C. (1994). Bernstein. Ih (1994). Psychometric theory, 3.
Powers, D. E. (2010). Validity: What does it mean for the TOEIC tests. ETS Research Rep. No. TC10, 1.
Roever, C., & McNamara, T. (2006). Language testing: The social dimension. International Journal of Applied Linguistics, 16(2), 242-258.
Rossiter, J. R. (2008). Content validity of measures of abstract constructs in management and organizational research. British Journal of Management, 19(4), 380–388.
48
Sireci, S. G., & Geisinger, K. F. (1992). Analyzing test content using cluster analysis and multidimensional scaling. Applied Psychological Measurement, 16(1), 17- 31.
Smith, M., & Smith, T. (2004). Introduction to educational administration: Standards, theories, and practice. Upper Saddle River, NJ: Pearson Merrill Prentice Hall.
Stanley, J. C., & Hopkins, K. D. (1972). Educational and psychological measurement and evaluation. New Jersey: Prentice-Hall Inc.
Taber, K. S. (2018). The use of Cronbach’s alpha when developing and reporting research instruments in science education. Research in science education, 48, 1273-1296.
Thorndike, R. M., Cunningham, G. K., Thorndike, R. L., & Hagen, E. P.
(1991). Measurement and evaluation in psychology and education. Macmillan Publishing Co, Inc.
Tran, H. M., & Pham, T. D. (2019). Expert review of VSTEP test items: Implications for content validity. Vietnamese Journal of Language Assessment, 5(1), 32-46.
Trochim, W. M., & Donnelly, J. P. (2008). Research methods: The concise knowledge base. Cengage Learning.
Valentin, J. D. (1996). Assessing Content-Related Validity and Internal-Consistency Reliability of Tests Constructed by Seychellois Teachers. Edith Cowan University. Retrieve from https://ro.ecu.edu.au/theses_hons/733
Van Dijk, T. A., Kintsch, W., & Van Dijk, T. A. (1983). Strategies of discourse comprehension. New York, NY: Academic Press.
Weir, C. J. (2005). Language Testing and Validation: An Evidence-Based Approach.
Palgrave Macmillan.
Woolley, G. (2011). Reading Comprehension. Reading Comprehension, 15–34.
https://doi.org/10.1007/978-94-007-1174-7_2
Worthen, B. R., Borg, W. R., & White, K. R. (1993). Measurement and evaluation in the schools. New York: Longman Publishing Group.
I
APPENDICES
Appendix 1. Description of the reading test
Part Description Question in
the test 1 Three third-level questions (low-level) of the following sub-
skills:
Determine the reference right before or after a reference, with little or no distracting elements.
Identify specific and clear information, information expressed in vocabulary and structure at level 3.
Locate specific information (in which paragraph)
Two third-level (average) questions of the following sub- skills:
Find and understand specific information in certain paragraphs of text (level 3 or 4).
Recognize clear information expressed differently from
the original text.
One third-level question (high) of the following sub-skills:
Understand the main meaning of a text.
Identify complex references in the text (with distracting elements).
Understand the important meanings of the article (vocabulary, structure at level 3 or 4; readings with texts and clear texture on relatively familiar topics).
Two fourth-level (low-level) questions of the following sub-skills:
Guess the meaning of new words in context.
Identify the viewpoint, the attitude expressed by clear information.
Identify the main content/purpose of the entire text (fourth-level passage, clearly structured and reasoned).
One fourth-level (average) question of the following sub- skills:
Understand the purpose, function of a clearly structured text.
Question 2 Question 4 Question 5 Question 6 Question 9
Question 3
Question 7 Question 8
Question 1
II
Part Description Question in
the test
Find and understand information (specific or not) – information that is scattered throughout the text (requires reading the entire text, text with less familiar topics).
Identify a secondary idea for a thesis or a key idea (vocabulary, structure at fourth-level; readings with more familiar themes).
One fourth-level question of the following sub-skills:
Recognize details/ information interpreted in a different way (with implicit discourse elements)
Identify the position, the viewpoint, the author's intentions (Implicit discourse)
Question 10
2 One third-level question (low-level) of the following sub- skills:
Determine the reference right before or after a reference,
with little or no interference.
Identify specific information, clear information, information expressed in vocabulary and structure at level 3.
Position specific information (in which paragraph).
Two third-level (average) questions of the following sub- skills:
Find and understand specific information in certain paragraphs.
Recognize clear information expressed differently from
the original text.
One third-level question (high) of the following sub-skills:
Identify complex references in the text (with distracting elements).
Understand the important ideas in the article (vocabulary and structure at level 3 or 4; readings with clear texts and texts on relatively familiar topics).
One fourth-level question (low-level) of the following sub- skills:
Guess the meaning of new words in context.
Identify the viewpoint, the attitude expressed by clear information.
Question 15
Question 11 Question 16
Question 14
Question 13
III
Part Description Question in
the test
Defines the structure or organization of clearly structured text.
Understand the main meaning of a paragraph (fourth-level
text with less familiar topics).
Two fourth-level (average) questions of the following sub- skills:
Understand the purpose, function of texts that are structure clearly.
Find and understand information (specific or not) – information that is scattered throughout the text (requires reading the entire text, text with a less familiar topic).
Understand the logic of sentences in text based on linking tools (references, references, synonyms, repetitions, etc.).
Identify complex references (fourth-or fifth-level texts with relatively complex texture and argumentation, on
unfamiliar topics).
Identify a secondary idea for a thesis or a key idea (vocabulary and structure at fourth-level; familiar topic readings).
One fourth-level (high) question of the following sub-skills:
Recognize details/ information interpreted in a different way (with implicit discourse elements).
Identify the main content or purpose of the entire text (fourth-level text, complexly structured and argued on
unfamiliar topics).
Identify the position, the viewpoint, the author's intentions
(discourse elements).
Understand the general tone of the entire text (four-or fifth- level texts with relatively complex texture and argumentation, on less familiar topics).
Two fifth-level questions (low-level) of the following sub- skills:
Identify the meaning of a sentence/a detail.
Identify the purpose of an information or argument.
Assume the meaning of a word in a context.
Question 12 Question 19
Question 17
Question 18 Question 20
IV
Part Description Question in
the test
Recognize details/information interpreted in a different way (with syntax elements, vocabulary used at level 5) 3 One third-level (average) question of the following sub-
skills:
Find and understand specific information in certain paragraphs.
Recognize clear information expressed differently from
the orginal text.
Two third-level (high) question of the following sub-skills:
Identify complex references in the text (with distracting elements).
Understand the main meaning of a text.
Identify complex references in the text (with distracting elements).
Understand the important meanings of the article (vocabulary, structure at level 3 or 4; readings with texts and clear texture on relatively familiar topics).
One fourth-level question (low-level) of the following sub- skills:
Guess the meaning of new words in context.
Identify the viewpoint, the attitude expressed by clear information.
Understand the main meaning of a paragraph (fourth-level
text with a less familiar topic).
One fourth-level (average) question of the following sub- skills:
Find and understand information (specific or not) – information that is scattered throughout the text (requires reading the entire text, text with a less familiar topic).
Understand the logic of sentences in text based on linking tools (references, references, synonyms, repetitions, etc.).
Identify complex references (fourth- or fifth-level texts with relatively complex texture and argumentation, on
unfamiliar topics).
Question 22
Question 25 Question 28
Question 24
Question 29
V
Part Description Question in
the test Two fourth-level questions of the following sub-skills:
Recognize details/ information interpreted in a different
way (with discourse elements).
Identify a secondary idea for a thesis or a main idea (fourth- or fifth-level text with a less familiar topic).
Identify the main content or purpose of the entire text (fourth- or fifth-level text with more complex structures and
arguments on less familiar topics).
Identify the position, the viewpoint, the author's intentions (discourse).
Understand the general tone of the entire text (fourth- or fifth-level texts with relatively complex texture and argumentation, on less familiar topics).
Two fifth-level questions (low-level) of the following sub- skills:
Identify the meaning of a sentence/ a detail.
Identify the purpose of an information or argument.
Assume the meaning of a word in a context.
Recognize details/informations interpreted in a different way (with meaningful elements, vocabulary used at level 5).
One fifth-level (average) question of the following sub- skills:
Exactly understand a subtle/complex detail of attitude,
reasoning, opinion in the text.
Recognize details/informations interpreted otherwise (expressions in the original text, repressions and/or 5-level
vocabulary and expression options).
Find and understand information – information that is scattered throughout the text (requires reading the entire text, highly specialized text at level 4 or 5).
Question 23 Question 26
Question 27 Question 30
Question 21
4 Two third-level questions of the following sub-skills:
Identify complex references in the text (with distracting elements).
Find and understand specific information in certain paragraphs.
Question 35 Question 36
VI
Part Description Question in
the test One fourth-level question (low-level) of the following sub-
skills:
Guess what new words mean in context.
Identify the viewpoint, the attitude expressed by clear information.
Two fourth-level questions of the following sub-skills:
Recognize details/information interpreted in a different
way (with discourse elements).
Identify a secondary idea for a thesis or a main idea (fourth- or fifth-level text with a less familiar topic).
Identify the main content or purpose of the entire text (fourth- or fifth-level text with more complex structures and
arguments on less familiar topics).
Identify the position, the viewpoint, the author's intentions (discourse)
Understand the general tone of the entire text (fourth- or fifth-level texts with relatively complex texture and argumentation, on less familiar topics).
One fifth (low level) question of the following sub-skills:
Identify the meaning of a sentence/a detail.
Identify the purpose of an information or argument.
Assume the meaning of a word in a context.
Recognize details/informations interpreted in a different way (with meaningful elements, vocabulary used at level 5).
Two fifth-level (average) questions of the following sub- skills:
Exactly understand a subtle/complex detail of attitude,
reasoning, opinion in the text.
Recognize details/information interpreted otherwise (expressions in the original text, repressions and/or fifth- level vocabulary and expression options).
Find and understand information – information that is scattered throughout the text (requires reading the entire text, highly specialized text at level 4 or 5).
Question 34
Question 31 Question 32
Question 33
Question 37 Question 38
VII
Part Description Question in
the test Two fifth-level (higher) questions of the following sub-
skills:
Identify views, attitudes expressed verbally.
Identify the meaning of a detailed sentence (fifth-level text is complex and/or highly specialized).
Find and understand information – information that is scattered throughout the text (requires reading the entire text, specialized text or high-level literature 5).
Understand the logic of an argument
Question 39 Question 40
VIII
Appendix 2. The reading sample test published by National Testing Center of Educational Testing and Assessment
IX
X
XI
XII
XIII