Using the DASH Method to Measure Reading Comprehension

Using the DASH Method to Measure Reading Comprehension WENYUH SHIEH Minghsin University of Science and Technology Hsinchu, Taiwan, Republic of China MARK R FREIERMUTH Gunma Prefectural Women’s University Gunma, Japan Vocabulary knowledge has been recognized by researchers as a critical component of reading comprehension, not only as a means to facilitate first language acquisition but also as an integral element in the learning of English as a second or foreign language To reduce the effect of insufficient vocabulary knowledge during reading, a compensatory strategy for many English as a foreign language (EFL) readers is dictionary consultation Several studies have investigated the effect of dictionary use on second language reading comprehension; however, findings have been inconclusive It is our contention that the reasons behind the inconsistent research findings stem from different manipulations of the related variables influencing reading comprehension To help moderate the effect of these variables, an innovative experimental approach, the duplicated answer sheets (DASH) method, was designed with the purpose of investigating the influence of dictionary use on text comprehension using a timed-interval assessment With this in mind, the DASH was given to five proficiency-leveled groups of EFL students at a university in Taiwan The groups were further divided into participants who used dictionaries and those who did not The results indicate that dictionary use has little effect during the early stages of testing However, during subsequent test intervals dictionary use seems to benefit learners’ text comprehension, regardless of proficiency level This research provides strong evidence that second language learners, given enough time, will benefit from using dictionaries as a tool to assist in text comprehension doi: 10.5054/tq.2010.217676 eading comprehension has always been a core issue in second language teaching research, and, as such, it continues to warrant a significant amount of investigation One of the key factors affecting a reader’s capability to comprehend a text is vocabulary knowledge Studies on text readability indicate that difficulties comprehending R 110 TESOL QUARTERLY Vol 44, No 1, March 2010 vocabulary can account for up to 80% of predicted reading variance (Alderson, 2000) Moreover, researchers have posited that a learner’s vocabulary knowledge can be correlated predictably to performance for reading comprehension tasks (see Crow, 1986; Yorio, 1971) Anderson and Freebody’s (1981) research supports this notion; in their study, they found that a causal relationship exists between vocabulary knowledge and reading comprehension As they point out, learners’ vocabulary knowledge is an important indicator of their language proficiency and an independent predictor of students’ text comprehension ability (cf Perfetti, 1985) Coady (1993) concurs, claiming that the more vocabulary knowledge one has the better reading comprehension ability one will have A common compensation strategy practiced by language learners lacking vocabulary knowledge is dictionary consultation Although this practice has been discussed in reading research, the studies aimed at finding the benefits or drawbacks of using dictionaries have produced a mixed bag of results The purpose of this study, then, is to investigate further the relationship between dictionary use and reading comprehension LITERATURE REVIEW Dictionary Use A review of research studies investigating the relationship between dictionary use and reading comprehension reveals a mix of advice from researchers Bensoussan, Sim, and Weiss (1984) question the benefit of using dictionaries in text comprehension In their study, three groups of advanced English as a foreign language (EFL) learners (i.e., monolingual dictionary group, bilingual dictionary group, and without dictionary group) took a battery of reading comprehension tests The authors concluded that dictionary use had little effect on comprehension and tended to slow learners down Participants who relied on dictionaries tended to be students with lower proficiencies; participants with higher reading proficiency scores tended to feel comfortable without using dictionaries Additionally, the researchers concluded that language learners, especially at lower proficiency levels, may not be particularly adept at using dictionaries They also suggest that students with lower reading proficiency levels feel a strong need to depend on dictionaries for compensation during test-taking situations, because they may not have the capability to apply contextual guessing strategies or they might not possess adequate vocabulary to draw inferences In light of these conclusions, Bensoussan et al suggest that teachers provide opportunities for students to improve their awareness of strategies, such USING THE DASH METHOD TO MEASURE READING COMPREHENSION 111 as identifying important words in a text and identifying the role of particular words in a text (i.e., the parts of speech) Partially replicating their study, Nesi and Meara (1991) draw a similar conclusion, namely, that dictionary consultation does not aid in text comprehension during reading tasks Hulstijn’s (1993) findings lend support to Nesi and Meara’s conclusions Hulstijn (1993) investigated intermediate EFL learners’ dictionary use and look-up behavior and concluded that there is no qualitative difference in reading comprehension performance on vocabulary tests By using multimedia annotations (text, picture + text, and video + text) for individual vocabulary items in second language reading, Chun and Plass (1996) also concluded that foreign language learners who look up more vocabulary items still not demonstrate better reading comprehension; however, a moderate correlation was found between vocabulary knowledge and reading comprehension Nevertheless, the waters remain clouded, because there are ample counterarguments extolling the benefits of using dictionaries as a compensatory reading aid strategy For example, Luppescu and Day (1993) point out that, when learners are not able to infer meaning from the context, dictionary consultation helps learners disambiguate word meaning (see also Fraser, 1999) Grabe and Stoller’s (1997) support stems from a five-month case study based on a Portuguese learner who used a bilingual dictionary for extensive reading Their conclusion is that dictionary use is extremely beneficial in learning a second language They argue that consulting a dictionary while reading not only provides the learner with a ‘‘psychologically valuable accuracy anchor’’ (Coady & Huckin, 1997, p 54) but also facilitates text comprehension Rumizen’s (1994) findings also support the notion that dictionary consultation appears to offer an advantage It seems particularly useful for decoding texts and as an effective tool for lexical access when using a bilingual dictionary during global-reading comprehension tests Also, Gonzales (1999) concluded from student interviews that dictionary use, though laborious, was still necessary Dictionary consultation ‘‘provides fast and reliable support for learners who have vocabulary and language limitations’’ (p 265) Reader and Reading Variables Before testing the effects of dictionary use as it relates to this study, it needs to be pointed out that the elements that comprise reading comprehension are rather complex (Chun & Plass, 1996) Though vocabulary knowledge plays a crucial role in reading comprehension, researchers have uncovered other factors, which might influence text 112 TESOL QUARTERLY comprehension as well Alderson (2000) categorizes these variables into two main types: reader variables and text variables Reader variables include those factors related to the readers themselves They are a reader’s (1) language proficiency level; (2) knowledge of the topic and the world; (3) purpose and motivation toward the reading task; (4) individual reading strategies; (5) affective state while reading; and (6) relatively stable characteristics, such as age, personality, gender, and social or educational background Text variables refer to the genre and the type of texts, such as the topic, content, and linguistic-related variables, such as lexical, syntactical, semantic difficulties, and text organization Research has also identified other factors that can have a profound effect on reading performance as well, and these include such elements as the reading environment, medium of text presentation (Kerr & Symons, 2006), typographical features (Anderson, 2003; Herman, Anderson, Pearson, & Nagy, 1987), and time available for reading Of these, the element of time has not been satisfactorily examined This is unfortunate, because the time duration of a reading test, as well as the overall length of the text are two relatively weighty matters, especially when one considers the reality that many university students must read extensively in the second language, outside of the classroom setting In light of this, it is of paramount importance to understand the effect duration has on a reading task, as it concerns a learner’s reading ability and text comprehension When examining previous research related to dictionary use, one can see that learners’ proficiency levels as well as time variables play crucial roles Some studies controlled for these variables; others did not Knight (1994) investigated the effect of dictionary access on a reading comprehension test, taking into consideration the learners’ different verbal abilities The results favor the benefits of dictionary use while reading—the low verbal ability group benefited significantly more from dictionary use while reading than did their high verbal ability counterparts Learners with dictionary access in each proficiency group spent more time in completing the test than their counterparts with no dictionary access In Luppescu and Day’s (1993) study, the researchers noted that, despite the benefits associated with dictionary use, there was a decrease in overall reading speed Nesi and Meara (1991) also found that dictionary users took longer, and even those who had the dictionaries but did not use them took longer—as long as those who actually used their dictionaries Those without dictionaries completed the reading task significantly more quickly Nesi and Meara concluded, however, that the dictionary users, though slower, did not gain any benefit toward greater comprehension, despite the additional time (see also Bensoussan et al., 1984) USING THE DASH METHOD TO MEASURE READING COMPREHENSION 113 PURPOSE Without conclusive evidence either for or against dictionary use by second language learners, the issue requires additional analysis from another perspective Specifically, there is a need to know the effects of dictionary use that considers reading comprehension on a variety of texts over an extended period of time Our purpose, then, is to examine reading comprehension in timed increments over an extended time period as a way of gauging whether reading with the help of dictionaries is more or less effective than reading without using them By acknowledging that text comprehension is affected by factors such as learners’ language proficiency level; text difficulty; learners’ relevant textual background knowledge; individual ability in inferencing, predicting, and guessing from the context; the available time for reading; syntactical complexity; and the vocabulary knowledge of the learners, we attempt here to reduce the effect of these variables on the research results Our approach is to analyze comprehension of reading texts via an innovative research design called the duplicated answer sheets (DASH) method The DASH method is distinctive, inasmuch as it allows researchers to test student performance (reading comprehension) using dictionaries in chunks of time, while still controlling for other factors The manner in which the DASH method controls these factors is discussed in the Materials section In addition to analyzing reading comprehension in timed increments, we intend to address the following research questions: Does dictionary use enhance reading comprehension? What is the actual effect of time on reading comprehension? What is the effect of time, if any, on differences in reading proficiencies? A side benefit of this study is that we consider a learner’s reading habits outside of the classroom, where, generally, most extensive reading in foreign languages takes place If it can be shown that the use of dictionaries is progressively more beneficial toward reading comprehension over time, dictionary use can be recommended to students as a tool, rather than discouraged METHODOLOGY Participants The participants in this study were 55 university sophomores from the same department in a Taiwanese university (an EFL context) All of the students had had at least years of English language learning experience Using their freshman English term grades as a placement determinant (based mainly on the results of quizzes, midterms, and final examinations), 114 TESOL QUARTERLY the participants were divided into two groups with similar proficiency levels, that is, the with-dictionary group (Y group with 25 participants) and the without-dictionary group (N group with 30 participants) These groups were then subdivided into smaller groups, based again on their freshman English term grades (Y1, Y2, Y3, Y4, Y5; and N1, N2, N3, N4, N5) As can be seen from Table 1, participants whose scores placed them in Y1 and N1 were the most proficient students, and students whose scores placed them in Y5 and N5 were the least proficient Table shows the divisions of the groups and subgroups based on their scores Materials To investigate the effect of dictionary use on reading among learners of different proficiency levels, we used a reading comprehension test with a multiple-choice format developed by a local test service center The format of the test was patterned after a nationwide General English Proficiency test used in measuring reading comprehension, with questions consisting of one correct item and three distractors Although there are some concerns when using a multiple-choice format in vocabulary tests and reading comprehension (see Nation, 1990; Peirce, 1992), it was used here for its overriding convenience of administration and relative ease of analysis (Wesche & Paribakht, 1996) The 50-item exam tested reading comprehension for the sentence level (connections and meaning within a sentence) and for longer passages (connections and meaning at the discourse level), in the form of vocabulary choice, cloze, and reading passages of general topics (see Appendix) The test scores were used to indicate the effect of dictionary use on the participants’ overall reading comprehension The DASH method was used to score the test, because it allows for incremental scoring over extended periods of time DASH Method as a Tool of Investigation The DASH method consists of multiple carbon-copy style answer sheets, which can be peeled off as a way to record student scores in timed TABLE Means of Term Scores of Y Group (With-Dictionary) and N Group (Without-Dictionary) Dictionary group (Number of participants) Y1/N1 (5)/(6) Y2/N2 (5)/(6) Y3/N3 (5)/(6) Y4/N4 (5)/(6) Y5/N5 (5)/(6) 86.2/85.8 78.1/77.6 71.8/71.3 65.6/66.4 45.0/51.3 5.23/9.1 1.56/2.6 1.48/1.4 2.10/1.4 15.26/14.6 USING THE DASH METHOD TO MEASURE READING COMPREHENSION 115 Mean of term scores Y/N Standard deviation Y/N increments In our case, participants’ score sheets consisted of six different answer sheets, so that the scores could be collected every 15 during the 90-min test (at the 15-, 30-, 45-, 60-, 75-, and 90-min marks) In other words, each answer sheet peeled off and collected represented the test score that a subject gained during the allotted test time Since participants were able to change their answers on previously answered questions, each test score was treated as a score independent of the other five that were collected.1 Regarding the rationale behind using the DASH method, as previously alluded to, the components affecting reading comprehension are multiple and complex (see Chun & Plass, 1996) In addition to learners’ language proficiency levels, there are still components such as syntactic structures, background knowledge, depth of vocabulary knowledge, and available reading time We employed the DASH method to effectively control for these variables In short, the benefits of employing the innovative DASH method as a measurement device are as follows No Textual Factors Involved; Clean Time Effect Data Variables such as textual knowledge and ease or difficulty of a text can affect readers’ performance on comprehension tests Investigating the time effect of dictionary use on learners of different proficiency levels using a traditional test method would require the researchers to give the participants six different tests (one requiring 15 to complete, another requiring 30 min, another requiring 45 min, etc) In addition, even with an extensive effort to modify such tests with the aim of achieving test similarity between the six versions, factors such as text difficulty and topical knowledge would likely have a considerably negative effect on the results It might be argued that, by employing identical tests repetitively, the text difficulty variance could be addressed However, researchers would then be faced with the practice effect caused by readers’ repeated exposure to the same texts and questions, not to mention the fact that this is not remotely similar to how reading is done or tested in more natural settings Additionally, if learners are indeed learning, they logically should score higher on subsequent tests And, regarding topical knowledge, although there are particular readability formulas, such as the Lix formula (Knight, 1994), there is always the problem posed by using different test texts, because different learners inevitably have different strengths as related to content 116 Observation of students revealed that occasionally students did change their answers Although the frequency of changed answers was not tabulated, it would make for an interesting study in its own right to see how often second language learners change answers on reading tests TESOL QUARTERLY (background) knowledge The DASH method allowed us to analyze test scores using the same texts in relation to the time effect By obtaining the mean scores of the participants in each collection, the factor of individual textual knowledge or background knowledge could thus be averaged out, leaving the scores and making for a clean time effect Time Efficiency and Practicality In addition to the benefit of the clean time effect associated with using the DASH method, the traditional method of giving six different tests at 15, 30, 45, 60, 75, and 90 min, respectively, would consist of testing participants for a total of 315 to complete such a study Irrespective of any variability in text difficulties, the diminishing returns on reliability and validity because of participants’ affective conditions (such as sheer exhaustion and boredom) and environmental stability in the over-5-hr, nonstop testathon would be significant By incorporating the DASH method in this study, only one reading assessment test was needed, because the other variables could be easily controlled (leaving the variable of time and learners’ proficiency levels to be investigated with dictionary use while reading) Besides the time-saving effect of the DASH method, it was also economical, because it uses duplicate carbon copies Procedure The participants were provided with the same testing material during the 90-min test Prior to the test administration, the participants were informed of the purpose of the study and reassured that their test scores would not be used as a determining factor in their English class grade This was done as a means to lower students’ reading anxiety while taking the test (Saito, Garza, & Horwitz, 1999), with the hope that they would, nevertheless, try their best to answer the questions The same 90-min test was given to both groups of students using the DASH method as a scoring tool (Figure 1) The test sections were collected as intended at the 15-, 30-, 45-, 60-, 75-, and 90-min marks The participants with dictionary access were allowed to choose the kind of dictionary they were familiar using and were asked on the answer sheet to indicate the type of dictionary they had used during the test: monolingual (English only), bilingual (English with equivalent Chinese translations), or a dual-bilingual dictionary that contains English entries with English definitions and their literal Chinese translations The participants were also told and encouraged to look up the unfamiliar vocabulary they encountered, to fulfill the aims of this study, and they USING THE DASH METHOD TO MEASURE READING COMPREHENSION 117 FIGURE The DASH method were required to underline the words they looked up Through the researchers’ on-site observation, informal posttest interviews with the participants in the dictionary group, and the underlined words on the question sheets, it was clear that the with-dictionary groups had consulted dictionaries extensively throughout the test as a means to identify vocabulary items with which they were unfamiliar RESULTS Table shows the average reading performance of the two groups at each time allotment The mean test scores of group Y (the withdictionary group) and group N (the without-dictionary group) in relation to the amount of time spent are recorded in Table and plotted in Figure 2, with the x-axis representing the amount of time spent and the y-axis representing the effect of dictionary use as determined by the test scores The table indicates that without-dictionary users scored TABLE Mean Scores of With-Dictionary Group Y and Without-Dictionary Group N in Relation to Time Spent Time spent (min) 15 Mean (SD) 30 Mean (SD) 45 Mean (SD) 60 Mean (SD) 75 Mean (SD) 90 Mean (SD) Mean scores of Y Mean scores of N 18.2 (6.3) 33.7 (11.4) 43.2 (12.6) 49.3 (13.6) 54.0 (14.6) 57.0 (14.8) 21.3 (11.3) 36.1 (15.0) 42.1 (15.5) 46.2 (13.4) 47.8 (14.1) 48.1 (13.8) Note SD standard deviation 118 TESOL QUARTERLY FIGURE Average reading performance higher during the early stages; however, at about 45 into the test, scores leveled; from that point forward, dictionary users performed increasingly better As seen in Figure 2, a sharp rise in test scores is seen in the first 60 of the test However, a much slower rate of gain was observed after the fifth collection, at the 75-min mark Toward the end of the 90-min test, a much slower rate in mean score increases was evident, especially in the without-dictionary group, where the difference between the mean scores recorded at 75 (47.8) and 90 (48.1) was, in practical terms, not significant During the first 15 min, the average score of the dictionary-use group proved lower than that of the without-dictionary group Nevertheless, as Figure illustrates, dictionary users recorded progressively higher scores on the reading test over time The crossover point circled in Figure indicates the approximate time at which dictionary use started to become more effective toward comprehending the text Table shows the mean scores of the five subgroups of the withdictionary (Y1 to Y5) and the without-dictionary groups (N1 to N5); these are plotted in Figure and Figure 4, respectively Except for the lowest proficiency groups Y5 and N5, the without-dictionary groups N1 to N4 seemed to outperform their dictionary counterparts Y1 to Y4 during the initial stages During the latter stages, however, dictionary use proved to be more effective on reading performance, as participants began tallying higher reading comprehension scores compared with the without-dictionary groups Moreover, the lowest proficiency group with dictionary use (Y5) consistently performed better than the withoutdictionary counterparts (N5) throughout the 90-minute test (Figure 5) USING THE DASH METHOD TO MEASURE READING COMPREHENSION 119 TABLE Mean Scores of Y1(N1) to Y5(N5) in Relation to Time Spent Time spent (min) Mean of Y1/N1 Mean of Y2/N2 Mean of Y3/N3 Mean of Y4/N4 Mean of Y5/N5 Mean scores of Y/N groups 15 30 45 60 75 90 23.2/32.3 20.0/26.3 18.0/21.3 16.0/17.0 14.0/9.7 18.2/21.3 42.0/50.3 38.8/43.3 36.0/38.7 26.0/28.0 25.6/20.0 33.7/36.1 53.6/56.3 50.8/47.0 44.4/42.3 36.4/35.0 30.8/26.3 43.2/42.1 61.6/60.7 57.2/49.7 49.2/47.0 43.2/40.3 35.2/33.3 49.3/46.2 66.8/62.7 63.6/49.3 52.8/48.0 47.2/40.3 39.6/34.7 54.0/47.8 70.4/61.3 66.4/50.0 56.0/48.7 50.0/44.3 42.0/35.0 57.0/48.1 FIGURE Test scores of dictionary groups (Y1–Y5) FIGURE Test scores of no-dictionary groups (N1–N5) 120 TESOL QUARTERLY FIGURE Test scores of with-dictionary group (Y5) and without-dictionary group (N5) DISCUSSION As can be seen in Table 2, in all instances but one, during the first few time intervals the with-dictionary groups did not as well as the without-dictionary groups However, as time progressed, the withdictionary group gradually showed better performance At the end of the 90 min, both groups’ scores began to level off However, as Figure clearly illustrates, the end results indicate that the with-dictionary groups consistently outperformed the without-dictionary groups One conclusion that may be drawn from these results is that, when language learners are given sufficient reading time, reading comprehension problems due to inadequate vocabulary knowledge can be compensated for to a certain degree by looking up unknown words In other words, by reducing the influences of text comprehension variables (e.g., background knowledge or textually related knowledge), dictionary use can be shown to be beneficial to readers of different proficiency levels over the long run We assume that, because dictionary use consumes time, the first four proficiency groups (Y1, Y2, Y3, and Y4) did not perform better than their counterparts in the without-dictionary group (N1, N2, N3, and N4) during the first half of the test; however, as the test continued, dictionary use became an increasingly valuable asset These results agree with Luppescu and Day’s (1993) conclusions, namely, that dictionary use is beneficial to overall text comprehension but it also causes decreases in reading speed USING THE DASH METHOD TO MEASURE READING COMPREHENSION 121 The fact that the without-dictionary group did better in the short term is an indication that scores might be related to usable time; in other words, because learners without dictionaries did not spend time looking words up, they might have kept answering questions using strategies such as finding contextual clues, identifying syntactical structures, or guessing, essentially allowing them to answer more questions in a shorter amount of time Another possibility is that they might have started out by scanning the texts, looking for relatively easy questions to answer, increasing the odds of getting more correct answers during the initial periods of the test time However, as test time progressed, and the remaining questions became increasingly difficult, reading comprehension would have also become increasingly difficult for the withoutdictionary group On the other hand, the dictionary users with sufficient test time may have been less concerned about scanning the texts, trusting their dictionaries to help them clarify the meanings of unknown words, and, consequently, to help them improve text comprehension Such frequent word consultation would undoubtedly slow down reading speed, resulting in lower scores in the short term but would prove beneficial when encountering more difficult items later on Certainly, this notion requires a more thorough investigation; nevertheless, it seems a reasonable explanation for the temporary better performance of the without-dictionary groups Grabe and Stoller (1997) mention the affective aspect associated with dictionary use Dictionary use represents a learner’s level of trust in a particular strategy choice Which reading strategies learners incorporate to help them comprehend texts is dependent on how strongly they believe in the effectiveness of a strategy (Jime´nez, Garcia, & Pearson, 1996) The looking up and slowing down behavior demonstrated by the with-dictionary group (Luppescu & Day, 1993) is not considered detrimental; rather, it reflects the beliefs (Kamhi-Stein, 2003) and reading habits of these and many other EFL learners With their psychological anchor at hand, a great many EFL readers will immediately turn to a dictionary when encountering a troublesome word—unfamiliar words equating with incomprehensible texts This misconception that dictionaries always help reading comprehension is not unusual among EFL learners To them, dictionary use seems to be the main solution toward resolving text comprehension issues The positive effect on readers’ text comprehension is valued at the expense of slower reading speeds associated with the time expended while looking up vocabulary This study also revealed that there is a limit to the benefit of dictionary use on reading comprehension Even with every unknown word looked up (with ample test time), readers’ text comprehension will ultimately become saturated, as the final scores of each group plateau No matter how much additional time is provided for the participants, the 122 TESOL QUARTERLY test scores will not continue to rise in proportion to the addition of time increments provided (Shieh, 2002) To be more specific, the saturation phenomenon observed in the study indicates that vocabulary knowledge influences text comprehension; the more words that a learner knows, the better his or her reading comprehension ability (Coady, 1993) Figure shows the expected long-term result of the effect of time on dictionary use The dotted line at the 90-min time slot indicates the total experimental time (minutes) in this study In brief, dictionary use, although certainly aiding text comprehension, can guarantee neither successful nor complete text comprehension Other factors are also involved, as can easily be observed (Figure 3) by the different final scores for each subgroup (see also Swaffar, Arens, & Byrnes, 1991) Figure shows the results of the comparisons of the two groups with the lowest proficiency in this study, that is, Y5 and N5 As seen in this figure, the scores of the with-dictionary Y group remain consistently higher than the without-dictionary N group, from the beginning of the test to the end of the 90 Because the participants in these two groups have the lowest language proficiency, it would be more difficult for them to benefit from other reading strategies, such as guessing from the context or identifying a particular syntactical structure They are more dependent than other groups on vocabulary knowledge, which confirms the findings of previous studies, indicating that readers of low FIGURE Expected performance with longer reading time USING THE DASH METHOD TO MEASURE READING COMPREHENSION 123 proficiency levels benefit more than readers at other proficiency levels when employing dictionaries (see especially Knight, 1994) Bensoussan et al (1984), as well as Nesi and Meara (1991) have concluded that dictionary use does not aid reading comprehension scores Knight (1994) argues for using dictionary consultation as a means to improve reading comprehension scores, especially for low-proficiencylevel learners Our results indicate that, when there is not sufficient time provided for the readers, dictionary consultation will not have an effect on reading performance (as is shown in Figure 2) Yet, when sufficient time is provided to participants for either short or long passages, dictionary use is beneficial toward readers’ text comprehension This is important, because foreign language learners read more in isolation than in tightly controlled, timed situations that are restricted primarily to the classroom and that are inevitably testlike, as was the case even in this study To sum up, dictionary use, especially in cases where language learners have ample time to read texts in the target language, should not be viewed in such a negative light; rather, teachers should encourage students to use dictionaries in such settings instead of discouraging their use SUMMARY AND CONCLUSIONS We concur with previous research indicating that vocabulary knowledge is one of the most reliable predictors of readers’ text comprehension; hence, in order to make up for an inadequacy of vocabulary knowledge, dictionary use while reading can be employed as a compensatory strategy To complement this proposition, we draw the following conclusions: N N N N N Reading time is an important factor in understanding text comprehension Test scores rise with longer reading time; however, there is a saturation effect Dictionary use benefits reading comprehension, especially for readers of low proficiency levels However, it is the case that reading time can be limited to the point where dictionary usage may be of little benefit Language learners of all levels will likely benefit from using dictionaries when they are expected to extensive reading The DASH method, which was developed for reading research, is an effective measurement tool for timed reading tests, especially when the researcher wants to look at the effect that time has on comprehension Based on the results of the beneficial effects of dictionary use on reading, it can be argued that learners need to be taught how to use a 124 TESOL QUARTERLY dictionary efficiently and effectively Moreover, in addition to strategies of dictionary use, learners also need to be taught other reading strategies to increase their text comprehension ability Reading strategies, such as predicting, contextual guessing, ignoring, and inferencing (Chamot, 1996) will equip readers with the ability to develop an overall reading fluency (i.e., reading with adequate comprehension at an appropriate reading rate; Kennel, 2003) It is likely that in many settings learners are not particularly adept at using reading strategies (simply because of a lack of opportunities to practice using them), making the teaching of such strategies vitally important As Fraser (1999) points out, dictionary use in combination with other strategies can facilitate second language reading performance to the greatest positive effect Limitations One limitation of the study is that there were only 55 students participating with a small number of learners in each subgroup, five or six Though the present study has provided quantitative evidence indicating the benefits of dictionary use on EFL reading, a larger number of participants would provide more definitive results In addition, it should also be noted that this study was done with Chinese native speakers studying English in Taiwan It might be the case that Chinese-speaking university students are accustomed to relying on dictionaries as a means to successfully process text comprehension problems, and as such they are very efficient dictionary users The DASH method in this study was employed in a paper-and-pen classroom situation It should be pointed out that the DASH method is also applicable to reading tasks done with computers A computerized DASH version could provide more detailed information for data analysis at any time point during the experiment On the other hand, one of the admirable features of the DASH method is that it is very simple for teachers to administer and score, making it a better fit for the ordinary classroom Pedagogical Implications Although it is true that dictionary consultation when reading is discouraged by some educators (Gonzales, 1999), our study indicates that learners benefit from using dictionaries It is important to recognize that the bulk of reading homework assigned to EFL or English as a second language students is much more extensive than students generally encounter on reading tests As teachers, we are asking students to read passages, sections, stories, or chapters in their entirety Hence to USING THE DASH METHOD TO MEASURE READING COMPREHENSION 125 attempt to limit dictionary use in such situations is in a sense to attempt to limit learner comprehension In fact, it behooves us as languageteaching professionals to seriously consider teaching students to effectively employ dictionary use as an integral part of reading pedagogy, because many students are going to be using their dictionaries once they walk out the classroom door, regardless of the instruction method used by the teacher This is not to relegate other reading strategies as unimportant The more strategies learners can employ, the more successful they will be Rather, our study is simply meant to trumpet the notion that dictionary usage can be of great benefit to second language readers ACKNOWLEDGMENTS We thank the editors of TESOL Quarterly and the reviewers for their helpful suggestions while crafting this work We also acknowledge the financial support provided by the National Science Foundation of Taiwan, Republic of China (NSC 952411-H-159-001) THE AUTHORS Wenyuh Shieh is an associate professor in the Department of Applied Foreign Languages at Minghsin University of Science and Technology, Taiwan, Republic of China Her primary interest concerns the role of vocabulary knowledge in foreign language teaching and learning, particularly as it pertains to English as a foreign language reading and listening comprehension Mark R Freiermuth is a professor in the Department of International Communication at Gunma Prefectural Women’s University, Japan Besides his interests in reading research, he also devotes research energy to analyzing discourses of computer-mediated forms of communication and enjoys developing and implementing language learning simulations in the classroom His publications are a reflection of these interests REFERENCES Alderson, C (2000) Assessing reading Cambridge, England: Cambridge University Press Anderson, N J (2003) Scrolling, clicking, and reading English: Online reading strategies in a second/foreign language The Reading Matrix, 3, 1–33 Retrieved from http://www.amschool.edu.sv/mined/LAm%20Models/article.pdf Anderson, R., & Freebody, P (1981) Vocabulary knowledge In J Guthrie (Ed.), Comprehension and teaching: Research reviews (pp 77–117) Newark, DE: International Reading Association Bensoussan, M., Sim, D., & Weiss, R (1984) The effect of dictionary usage on EFL test performance compared with student and teacher attitudes and expectations Reading in a Foreign Language, 2, 262–275 126 TESOL QUARTERLY Chamot, A U (1996) Accelerating achievement with learning strategies Glenview, IL: Scott Foresman Addison Wesley Chun, D M., & Plass, J L (1996) Effects of multimedia annotations on vocabulary acquisition Modern Language Journal, 80, 183–196 Coady, J (1993) Research on ESL/EFL vocabulary acquisition: Putting it in context In T Huckin, M Haynes, & J Coady (Eds.), Second language reading and vocabulary learning (pp 3–23) Norwood, NJ: Ablex Publishing Coady, J., & Huckin, T (1997) Second language vocabulary acquisition Cambridge, England: Cambridge University Press Crow, J T (1986) Receptive vocabulary acquisition for reading comprehension Modern Language Journal, 70, 242–250 Fraser, A (1999) The role of consulting a dictionary in reading and vocabulary learning The Canadian Journal of Applied Linguistics, 2, 73–89 Gonzales, R (1999) Building vocabulary: Dictionary consultation and the ESL student Journal of Adolescent & Adult Literacy, 43, 264–270 Grabe, W., & Stoller, F L (1997) Reading and vocabulary development in a second language: A case study In J Coady & T Huckin (Eds.), Second language vocabulary acquisition (pp 98–122) Cambridge, England: Cambridge University Press Herman, P A., Anderson, R C., Pearson, P D., & Nagy, W E (1987) Incidental acquisition of word meaning from expositions with varied text features Reading Research Quarterly, 22, 263–284 Hulstijn, J (1993) When foreign-language readers look up the meaning of unfamiliar words? The influence of task and learner variables Modern Language Journal, 77, 139–147 Jime´nez, R., Garcı´a, G., & Pearson, D (1996) The reading strategies of bilingual Latina/o students who are successful English readers: Opportunities and obstacles Reading Research Quarterly, 31, 90–113 Kamhi-Stein, L (2003) Reading in two languages: How attitudes toward home language and beliefs about reading affect the behaviors of ‘‘underprepared’’ L2 college readers TESOL Quarterly, 37, 35–71 Kennel, P (2003, March) What’s at the bottom: Improving the reading rate of ESL students Paper presented at the TESOL Convention, Baltimore, MD Kerr, M A., & Symons, S E (2006) Computerized presentation of text: Effects on children’s reading of informational material Reading and Writing, 19, 1–19 Knight, S (1994) Dictionary use while reading: The effects on comprehension and vocabulary acquisition for students of different verbal abilities Modern Language Journal, 78, 285–299 Luppescu, S., & Day, R R (1993) Reading, dictionaries, and vocabulary learning Language Learning, 43, 263–287 Nation, I S P (1990) Teaching and learning vocabulary New York, NY: Newbury House Nesi, H., & Meara P (1991) How using dictionaries affects performance in multiple-choice EFL tests Reading in a Foreign Language, 8, 631–643 Peirce, B N (1992) Demystifying the TOEFL reading test TESOL Quarterly, 26, 665–689 Perfetti, C A (1985) Reading ability New York, NY: Oxford University Press Rumizen, M C (1994) The effect of bilingual dictionary use upon the assessment of second language reading comprehension in Russian(Doctoral dissertation) Retrieved from ProQuest Digital Dissertations (AAT 9421564.) Saito, Y., Garza, T., & Horwitz, E K (1999) Foreign language reading anxiety Modern Language Journal, 83, 202–218 Shieh, W (2002, May 21) Test scores not correlate proportionally with test time The China Times [Taipei], p 8A Swaffar, J., Arens, K., & Byrnes, H (1991) Reading for meaning: An integrated approach to language learning Englewood Cliffs, NJ: Prentice Hall USING THE DASH METHOD TO MEASURE READING COMPREHENSION 127 Wesche, M., & Paribakht, T S (1996) Assessing second vocabulary knowledge: Depth vs breadth Canadian Modern Language Review, 53, 13–39 Yorio, C (1971) Some sources of reading problems for foreign language learners Language Learning, 21, 107–115 APPENDIX Sample Test Questions I Sentence completion: Choose the most appropriate word to complete each of the following sentences His parents passed away when he was only one year old So he was sent to an and grew up there (A) ordeal (B) orphanage (C) orbit (D) ordinal II Vocabulary in context: Choose the word that is closest in meaning to the underlined word She has been confined to a wheelchair for the rest of her life because of the accident (A) returned (B) restricted (C) devoted (D) decided III Cloze test Body temperature can affect how happy or unhappy we are when we wake up During the (31) of a day, our body temperature rises and falls at regular times Although we not (32) the change, it does affect our sleeping patterns When body temperature is up, we are awake As it (33) , we grow tired and, eventually we sleep As a result, anyone who has a fast-rising temperature (34) is a ‘‘morning person.’’ On the other hand, an ‘‘evening person’’ has a body temperature that rises slowly It doesn’t hit its high (35) until mid-afternoon, when this person feels best 31 32 33 34 35 (a) (a) (a) (a) (a) course attract presents thought point (b) (b) (b) (b) (b) cloud interest falls cycle deal (c) (c) (c) (c) (c) weight notice rolls mature tool (d) (d) (d) (d) (d) contest depend completes comment treat IV Reading Comprehension Human beings tasted chocolate first as a bitter drink A thousand years later, it became the sweet solid candy we know today The cocoa bean was first used by the Indian civilizations of South America to make a ceremonial drink which the Aztecs called ‘‘bitter water.’’ Columbus brought cocoa back to Spain in the 16th century It was mixed with large amounts of sugar and became a popular drink with King Ferdinand From Spain, the popularity of the chocolate drink spread first to Italy and France, and then to England Chocolate remained a drink until 1847 when the English made the first sweet-tasting, solid chocolate 48 What was one very early use of chocolate? (a) It was a popular drink (b) It was a health drink (c) It was a drink at ceremonies (d) It was a liquid medicine 49 Which country made the first dark, solid chocolate? (a) Spain (b) Italy (c) France (d) England 50 Which statement is NOT true? (a) The cocoa bean was first used by Indian civilizations to make a drink (b) Cocoa was introduced to Europe by King Ferdinand (c) The cocoa bean was first used by Indian civilizations to make a drink (d) Chocolate was first tasted as a bitter drink 128 TESOL QUARTERLY