The effects of text based and audio based dynamic glosses on l2 vocabulary learning a dynamic assessment approach

Dynamic glosses were operationalised as a set of incrementally ordered prompts, provided during text-based and audio-based interactions that guided the participants to identify the meani

Full Terms & Conditions of access and use can be found at

The Language Learning Journal

ISSN: (Print) (Online) Journal homepage: www.tandfonline.com/journals/rllj20

The effects of text-based and audio-baseddynamic glosses on L2 vocabulary learning: adynamic assessment approach

Ehsan Rassaei

To cite this article: Ehsan Rassaei (2023) The effects of text-based and audio-based dynamic

glosses on L2 vocabulary learning: a dynamic assessment approach, The Language Learning Journal, 51:4, 509-522, DOI: 10.1080/09571736.2023.2213247

To link to this article: https://doi.org/10.1080/09571736.2023.2213247

Published online: 13 Jun 2023.

Submit your article to this journal

Article views: 227

View related articles

View Crossmark data

The effects of text-based and audio-based dynamic glosses on L2 vocabulary learning: a dynamic assessment approach

Ehsan Rassaei

Faculty of English Language Studies, Majan University College, Muscat, Oman

The present study investigated the effects of text-based and audio-based dynamic glosses on L2 vocabulary learning within a sociocultural approach Dynamic glosses were operationalised as a set of incrementally ordered prompts, provided during text-based and audio-based interactions that guided the participants to identify the meaning of unknown vocabulary Fifty-seven EFL learners were assigned to three experimental groups – dynamic text-based, dynamic audio-based and traditional glosses– as well as a control condition The participants of the two dynamic glossing conditions received dynamic glosses via written‘chat’ or orally for a number of unknown words while reading a text In addition to a non-dynamic post-test, a computerised dynamic assessment tool was also designed to investigate learners’ mediated and unmediated knowledge of the target words following the treatment sessions The results provided evidence for the efficacy of both text-based and audio-based dynamic glosses for L2 vocabulary learning The analysis of learners’ mediated and unmediated scores calculated by the computerised tool also revealed that the learners’ mediated scores provide valuable information for assessing learners’ vocabulary knowledge In particular, the findings suggested that text-based dynamic glosses could result in higher mediated scores compared to the audio-based dynamic glosses.

Vocabulary learning is an essential aspect of L2 development, and an increasing number of studies have investigated different techniques for teaching L2 vocabulary to learners (e.g Candry et al.2020; Huang and Lin2014; Nassaji and Tian2010; Nguyen and Boers 2019; Sun2017; Zou2017) It is believed that a large proportion of new L2 vocabulary is learned during reading comprehension (Huckin and Coady1999; Rott et al.2002) However, unfamiliar words in a text may pose difficulty for L2 learners and disrupt reading comprehension activities (Nassaji2003) One way to help learners overcome their difficulty with unfamiliar words during a reading task is to add marginal glosses to unfamiliar words in a text (Kim et al 2020) A gloss can be defined as ‘a brief definition or synonym, either in L1 or L2, which is provided with the text’ (Nation2013, 238).

Providing glosses for unknown words in a text is a traditional technique for helping learners understand the meaning of unknown words Glossing is considered as among the most effective techniques for supporting L2 reading comprehension as well as vocabulary learning (Gettys et al 2001; Zhang and Ma2021) However, since traditional marginal glosses tend to provide L2 learners just with the definitions of target words in a text, they may not induce optimal cognitive involvement with the new vocabulary, which is arguably required for L2 development and vocabulary learning

CONTACTEhsan Rassaeiehsan.rassaei@majancollege.edu.om2023, VOL 51, NO 4, 509–522

https://doi.org/10.1080/09571736.2023.2213247

(Hulstijn and Laufer,2001) An alternative could be to present L2 learners with‘dynamic glosses’ (Rassaei2020) or incrementally ordered hints with the most implicit presentedfirst, leading gradually to the most explicit, in such a way as to help the learner identify the meaning of an unknown word with the minimum assistance necessary This can be seen as a form of mediation based on the prin-ciples of dynamic assessment (DA) and sociocultural theory Because dynamic glossing (DG) requires incremental interaction between a learner and a mediator, it can be implemented effectively through digital media.

Dynamic glosses can be provided in different modalities such as audio-based or text-based interactions Previous research has indicated that different modalities of interaction provide different affordances for language learning (e.g Blake2009; Fuente2003; Liao2018; Salaberry 2000; Yanguas 2012) The text-based mode of interaction has some unique advantages com-pared to other modes of interaction such as face-to-face and interaction with audio responses Such benefits include providing students with more simplified input and allowing more proces-sing and planning time, and in a less threatening context (Freiermuth and Huang 2012; Freiermuth and Jarrell 2006; Hsu 2017; Sauro 2009) Audio-based interaction, on the other hand, is similar to face-to-face interaction and allows for the inclusion of paralinguistic features such as intonation, stress and other suprasegmental features Little is known, however, about the differential effects of text-based and audio-based dynamic glosses on L2 vocabulary learning.

The aim of the present study is thus to investigate the effect on L2 vocabulary learning of different kinds of glossing in L2 reading texts, including audio-based dynamic glosses, text-based dynamic glosses and traditional non-dynamic glosses.

Glosses and vocabulary learning

Glossing has traditionally been considered an effective way of supporting learners faced with unfa-miliar vocabulary during L2 reading Previous studies provided evidence that different forms of glosses such as paper-based (e.g Chen 2002; Hu et al 2014; Miyasako 2002) and technology-mediated glosses (e.g Al–Seghayer2001; Chun and Plass1996; Nagata1999; Rassaei2018; Yoshii 2006) can promote L2 vocabulary learning There are different explanations for why glosses are ben-eficial for vocabulary learning First, glosses promote comprehensible input and meaningful learning and also help learners avoid incorrect and misleading guesses (Gettys et al.2001; Mohsen and Bala-kumar2011) Moreover, glosses can facilitate vocabulary learning by triggering different levels of noticing and cognitive processing (Yanguas2009) as well as enhancing form/meaning mapping (Nation2003) Therefore, the question has shifted from whether or not glosses are effective for voca-bulary learning to which form of glossing is more beneficial to facilitate L2 vocabulary during reading (Mohsen and Balakumar2011; Yoshii2006).

In recent years, researchers have also increasingly become more interested in studying the effects of computer-mediated multimedia glosses in vocabulary learning (e.g Al–Seghayer 2001; Boers et al.2017; Chun and Plass1996) In addition, previous research has also investigated the effects on L2 vocabulary learning of single versus multiple choice glosses (e.g Rott et al 2002; Watanabe1997) and L1 vs L2 glosses (e.g Chen 2002; Choi2016; Miyasako2002; Yoshii 2006) A meta-analysis performed by Kim et al (2020) revealed that L1 glosses are more effective than L2 glosses for vocabulary learning and that the effectiveness of glosses can be affected by learners’ proficiency level Another recent meta-analysis conducted by Ramezanali et al (2020) indicated that adding an additional mode to single glosses could enhance vocabulary learning whereas mul-tiple glosses may not necessarily enhance vocabulary learning outcomes The findings also revealed that the beneficial effects of glosses can be affected by different variables such as text type and learners’ proficiency level.

In general, while the studies reviewed above provided evidence that different ways of glossing can help the development of L2 vocabulary, there are some concerns regarding the use of glosses First, the

beneficial effects of glosses on L2 vocabulary learning during reading depend on several factors includ-ing learners’ overall proficiency, the frequency of unknown target words in the text and learner’ voca-bulary knowledge (Abraham2008) More importantly, while glosses can help learners understand the meaning of unknown words which can lead to vocabulary learning (Yoshii2006), they may reduce the level of cognitive processing and task engagement which is necessary for optimal learning This is because marginal glosses do not engage learners in a deep and elaborative processing to discover the meaning of unknown words (Hulstijn and Laufer,2001) Furthermore, SLA researchers commonly believe that interactive and collaborative learning environments are essential for successful L2 acqui-sition However, traditional marginal glosses do not facilitate vocabulary learning in an interactive manner Dynamic glosses (Rassaei2020) address the limitations of marginal glosses by inducing lear-ners’ engagement with the vocabulary learning task and pushing learners to identify correct definitions for unfamiliar vocabulary items in an interactive manner based on the principles of DA.

Dynamic assessment and dynamic glosses

DA is a framework for assessing learners’ potential level of development derived from Vygotsky’s sociocultural theory (e.g Vygotsky 1978;1987) The theory conceptualises development in terms of moving from dependence on external sources of mediation such as experts’ assistance, towards independent performance Aljaafreh and Lantolf (1994) defined mediation in response to learners’ errors as assistance which is graduated, dialogic and tailored to learners’ Zone of Proximal Development (ZPD) The concept of ZPD can be captured in terms of a learner’s mediated perform-ance or the level of mediation required by a learner to complete a task.

DA researchers are typically interested in assessing learners’ performance within their ZPD, that is, what they can do when they receive mediation (Davin 2013) Dynamic glossing is implemented based on the DA framework During a typical DA session, the mediator promotes learners’ develop-ment by providing mediation which is tuned to learners’ ZPD and at the same time documents and analyses learners’ responses to mediation to get insight into learners’ mediated performance In this way, DA tries to capture the learning process rather than simply focus on the product of learning Similarly, dynamic glosses can capture learners’ ZPD by providing incremental hints – from more implicit to more explicit– to help learners identify and learn the meaning of unknown words The DA framework and dynamic glosses thus serve both teaching and assessment purposes by providing incremental hints which are tailored to learners’ ZPD, allowing evaluation the the level of mediation required by a learner to complete a task as well as documenting and assessing the learner’s responses to mediation.

There are two main approaches for measuring development during DA Thefirst approach capi-talises on examining the explicitness of mediation that learners require to complete a task over time (e.g Aljaafreh and Lantolf 1994; Davin2013; Poehner and Lantolf 2013) One seminal paper that incorporated this approach is Aljaafreh and Lantolf (1994) The researchers designed a regulatory scale that included a number of prompts incrementally ordered from the most implicit to the most explicit The authors operationalised and reported learners’ microgenetic development in terms of their ability to complete a task, requiring incrementally less explicit prompts over time In a computerised DA study, Poehner and Lantolf (2013), examined the effects of an online DA tool on Chinese and French L2 reading comprehension The tool provided the learners with a number of multiple-choice items When learners chose incorrect options, the tool presented them with incre-mentally ordered hints from implicit to explicit By analysing the level of mediation the participants required to identify the correct option, the tool generated three values for learners’ performance: their actual, mediated and potential scores The authors argued that this tool could be administered to a large number of test-takers and could provide useful information about learners’ ZPD.

The second approach to measuring development, as adopted by several DA researchers (e.g Ableeva 2008, 2018), is to examine learners’ reciprocity during DA sessions Rassaei (2023), for example, used five reciprocity criteria to analyse learners’ responsiveness to mediation during

three mobile-mediated DA sessions During these sessions, the participants were presented with several scenarios that required them to make requests and received implicit to explicit mediation to improve their utterances when they failed to make appropriate requests Analysing the reciprocity criteria revealed that learners progressively became more responsive to mediation, and conse-quently achieved a more agentive role during the interactions Within the sociocultural theory, achieving a more agentive role in interaction suggests a more autonomous performance and is therefore an indication of development (Ableeva 2018) Thus, DA researchers and practitioners emphasise the role of assessing learners’ responsiveness to mediation because learners’ reciprocity in response to mediation during DA will provide valuable information about what learners can perform with assistance which in turn is an indicator of what the learners can perform independently in future (e.g Ableeva2018; Levi and Poehner2018; Poehner2008).

A number of previous studies have used a DA framework to investigate learners’ development regarding different aspects of L2 including reading comprehension, listening comprehension and grammar (e.g Anton 2009; Davin 2013; Ebadi and Bashir 2021) Regarding vocabulary learning, however, there has been little implementation of DA One notable study is Rassaei (2020) that inves-tigated the effect of text-based DA style glosses on L2 vocabulary learning by Iranian EFL learners during mobile-mediated interaction and provided evidence that dynamic glosses were more effective than traditional glosses Torang and Weisi (2023) replicated Rassaei’s study and similarly found that text-based dynamic glosses were more effective than non-dynamic glosses during mobile-mediated DA Alcaraz Mármol (2021) also compared the impacts of text-based dynamic glosses with traditional glosses and reported the advantage of dynamic over traditional glosses.

The present study

While previous research has provided evidence for the facilitative role of both traditional paper-based and computer-delivered glosses in L2 vocabulary learning, little attention has been paid to the impact of dynamic glossing within a sociocultural framework which emphasises maximising lear-ners’ reciprocity and their agency in accomplishing a collaborative activity Moreover, previous studies only investigated the effects of text-based dynamic glosses on L2 vocabulary learning The present study builds on the DA framework to investigate the effects of both text-based and audio-based dynamic glosses on L2 vocabulary learning To this end, the following research ques-tions guide the present study:

(1) Is text-based dynamic glossing (TDG) effective for L2 vocabulary learning in terms of remember-ing the meanremember-ing of unfamiliar words?

(2) Is audio-based dynamic glossing (ADG) effective for L2 vocabulary learning in terms of remem-bering the meaning of unfamiliar words?

(3) Are TDG and ADG more effective than non-dynamic glossing (NDG)?

(4) Is there a statistically significant difference between the effectiveness of TDG and ADG for L2 vocabulary learning?

The research Design

The study reported here was based on an experimental design Participants were randomly assigned to four groups: three experimental conditions (TDG group n = 14; ADG group n = 15; NDG group n = 14) and a control group (n = 14)

On thefirst day of the study, the participants took a classroom vocabulary pre-test where they were asked to provide a definition in their L1 or L2 of a list of 25 L2 words On the basis of this pre-test, the researcher was able to select 14 items, unknown to all participants, as targets for the

experimental treatment The two treatment sessions, where participants undertook L2 reading with different glossing of the target words (or no glossing), took place on days 2 and 4 via Google Meet Following each treatment session, a computerised DA was undertaken, as described below An immediate vocabulary post-test was administered on day 5, with a delayed test two weeks later The immediate and delayed post-tests were the same as the pre-test.

The participants were 57 intermediate level EFL learners studying English language at a university in Iran with Persian as their L1 They ranged in age from 21 to 28 and had studied the English language for about 5 years Participants were recruited from classes at the same academic level They were informed that participation in the study was entirely voluntary.

From the 73 students who originally volunteered for the study, 57 were selected on the basis of a reading comprehension test designed to ensure a homogenous group This test, taken from Rassaei (2019), comprised two passages of about 800 words, each followed by 10 multiple choice items Rassaei (2019) reported 0.72 and 0.75 reliability indices measured through KR-21 formula for two administrations of the test on a pilot group The KR-21 formula for the test as used in the present study indicated 0.75 internal consistency Students who scored 15 or above out of a maximum of 20 were recruited to the study Based on the reading comprehension test scores and background language learning information, as provided in the consent forms, the participants were judged to be at the B2 level of the Common European Framework of Reference (CEFR).

Target words

14 words which were not known to the participants of the study were selected based on participants’ performance in the pre-test The selected target words were as follows: extinct, pollute, vanish, affluent, replenish, habitat, exhaust, plunge, forage, expansive, appalling, perish, dwindle, deplete.

For the treatment sessions, six passages of around 200 words, adapted from Milson (2016), were used to present learners with the target words in an appropriate context Each text contained between four andfive target words The participants read three texts at each of the two treatment sessions; thus, they were exposed to all the target words twice over a period of three days The texts were carefully chosen to be at an appropriate level, with Flesch readability scores ranging between 60 and 70 (standard and average readability levels) No textual enhancement was made to the texts (including the target words) and no strict time limitation was imposed on the reading activity Each treatment session took place as a one-to-one computer-based meeting in Google Meet between the individual participant and an interlocutor who was a doctoral student in TEFL At each session, three texts were presented successively via the Google Meet screen.

TDG participants were instructed to read the texts and ask the meaning of any unknown words via text chat ADG participants were instructed to read the texts but ask for the meaning of the unknown words orally through their microphones In these two dynamic glossing conditions, the interlocutor/mediator then responded to the participant with a graduated series of responses – either in writing via the chat or orally, depending on the treatment condition– in order to help the participant identify the meaning of the unknown word.Table 1displays the mediation scale uti-lised to provide mediation to learners during the DG sessions The mediatorfirst began with the most implicit support, such as‘please read the sentence in which the word has been used and do your best to guess the meaning of the word’ If the learner was unable to come up with the meaning of the unknown word, the mediator increased mediation by providing more explicit prompts until the learner was able to correctly guess the meaning of the unknown word.

The following example indicates how DG was implemented in the text-based DG condition.

Learner:Replenish (sent via text-based chat to the mediator)

Mediator:(typing back) Please read sentence 3 and guess the meaning of replenish.Learner:[no response]

Mediator:The text states‘some countries USE resources FASTER that the NATURE can replenish them’.Learner:[no response]

Mediator:What does it mean to say‘when the resources are used and finished, they must be replenishedagain’

Learner:[no response]

Mediator:look at this example:‘when you exercise, you should drink a lot of water to replenish the water youhave lost’.

Learner:to replaceMediator:very good!

NDG participants were also instructed to read the texts and ask the meaning of the unknown words through text chat but in response, only a written L2 definition was given Participants in the control group simply read the three texts and had no opportunity for glossing help Both the NDG and the control groups were also asked to answer seven true/false questions on the content of the texts in order to ensure that time spent‘on-text’ was roughly equivalent for the four groups The target voca-bulary items were not included in the true/false questions The participants submitted their answers to the true/false questions in writing via the chat box The average time each participant spent on each text was between seven to nine minutes depending on the treatment conditions.

Test of target vocabulary

In order to measure learning of the target items, all participants took the same test three times: at pre-test, at post-test immediately after the treatment sessions, and at delayed post-test after two weeks As noted above, the vocabulary test consisted of 25 vocabulary items including the 14 target words that would be used in the treatment The 25 words were selected based on the researcher’s judgment that the participants of the study would not be familiar with these words based on their language proficiency Participants were simply required to give a definition in Persian or English for each word Based on the participants’ responses in the pre-test, 14 words which were totally unknown or least known by the participants were selected and used as the target words in the study With regard to the post- and delayed post-test, the same 14 target words plus 11 distractors were presented to the participants The distractors were different for the post-test and delayed post-test One point was scored for supplying a correct definition for each target word Spelling errors were ignored for L2 synonyms supplied by the learners as long as the word supplied could be identified as an acceptable synonym for the target word The total score for the vocabulary test was thus 14.

Dynamic assessment of learners’ development

In addition to the vocabulary test, a computerised test was designed to measure learners’ vocabulary development based on the DA framework The aim of this assessment instrument was to measure vocabulary development within learners’ ZPD; in other words, their mediated performance (i.e.

Table 1.Mediation scale used for DG.Mediation prompts

1 The learner is encouraged to read the sentence again and guess the meaning of the target word

2 The learner’s attention is drawn to a part of the text that provides clues for the learner to guess the meaning of the target word.3 A leading question that draws learner’s attention to the meaning of the target word is asked

4 The target word is used in a more transparent context.

what learners could work out regarding the meaning of the target words when they received mediation) The computerised DA test employed in the present study aimed to address a limitation of face DA, which is the relatively large amount of time which needs to be spent for face-to-face DA in language classrooms.

The test was designed as a software which could be installed on participants’ computers After installing the software through a download link, each participant received two texts in succession that included the target words for each DA session Each text of about 200 words included half of the target vocabulary items The texts were different from the ones used for the treatment sessions but were at the same proficiency level and were adapted from Milson (2016) Following each text, the participants were presented with the target words one by one and were instructed to identify a correct L1 equivalent for each word among seven options The seven options included one correct L1 equivalent and six distractors The learners were instructed to choose the correct response and press‘next’ to receive the next vocabulary item.

The major function of the computer tool that made it appropriate for DA was providing mediation and graduated support in response to learners’ incorrect answers If a learner selected an incorrect option, the tool provided the participant with four sequentially ordered prompts one by one, from most implicit to most explicit The prompts were different for each item The prompts were devel-oped based on the mediation scale used for preparing the prompts during the DG sessions The screenshot displayed inFigure 1indicates a participant’s performance on one of the computerised test items.

For each item, if the learner chose the correct answer at thefirst attempt without receiving any mediation, they were awarded 5 points If the learner was able to identify the correct definition after receiving thefirst mediating prompt, they scored 4 points and so on; only 1 point was awarded for a correct answer after the fourth prompt and no score was given if the correct answer could not be identified after all four prompts In that case, the tool provided the correct response and presented the learner with the next item A total‘mediated score’ was then calculated based on the average score out of 5 for each of the 14 vocabulary items (see Table 4) The tool also computed an

Figure 1.A computerised DA item Image depicting the Computerised DA item including a text with target words in bold-facedin the text There is one target word below the text with seven L1 equivalents.

‘unmediated’ score for each learner that represented the learner’s ability to identify the correct L1 equivalent of the target words in the absence of any mediation A score of 1 was assigned for every unmediated correct answer, with a possible total of 14 Based on the mediated and the unme-diated scores, the programme was able to quantify learners’ mediated performance and their poten-tial level of development Figure 2 displays how total mediated and unmediated scores were calculated for a learner for thefirst DA session The test-retest reliability of the computerised test was measured on a separate group of 37 advanced level EFL learners using KR-21 method The com-putation of reliability indicated 0.75 internal consistency.

Test of target vocabulary

Table 2displays the descriptive statistics for scores on the test of the 14 target vocabulary items in the four conditions Only the scores for the two post-tests are presented here as in all cases, pre-test scores were zero (only items which were entirely unknown at pre-test were selected as targets) All three experimental groups learned from the treatment, demonstrating higher levels of knowledge at post-test than the control group, and this knowledge was retained at delayed post-test Further, the two dynamic gloss groups outperformed the non-dynamic gloss group.

Due to the small sample size, the non-parametric Kruskal-Wallis H test was used to compare groups This indicated statistically significant differences among the groups in the post-test, H (3) = 44.3, p < 001, as well as the delayed post-test, H (3) = 43.4, p < 001.Table 3shows the post hoc multiple comparison results for the vocabulary post-test and delayed post-test, using Mann-Whitney U test with Bonferroni correction Effect sizes in terms of η2were also computed for the pair-wise group comparisons To estimateη2, the standardised group difference for each pair (i.e z score) was divided by the total number of participants minus one in the two groups This formula provides a robust measure of effect size for non-parametric data as well as continuous data with small sample

Figure 2.An example of the output provided by the DA tool Image indicating the level of mediation the learner required toanswer each item.

Table 2.Descriptive statistics for the vocabulary test.

size (Fritz, Morris, and Richler2012) The following benchmarks from Cohen (1988, 283–287) were used:η2= 0.01: small effect; η2= 0.06: medium effect, and η2= 0.14 a large effect.

AsTable 3indicates, no statistical difference was observed between the two DG groups (TDG and ADG) at either post- or delayed post-test However, there were statistically significant differences between the NDG group and each of the DG groups with large effect sizes There were also statisti-cally significant differences between each of experimental groups (TDG, ADG and NDG) and the control group with large effect sizes This confirms that all three treatment groups demonstrated greater vocabulary learning than the control, and that the two DG groups performed better the NDG group.

DA scores

Table 4presents the participants’ average mediated scores in the two computerised DA sessions Based on the scoring procedure detailed above, the maximum possible score for each individual in the computerised DA was 5.

AsTable 4indicates, learners’ mediated scores improved from the first to the second DA session The table also indicates that the two groups that received DA-style instruction during the treatment sessions improved most Learners’ higher mediated scores in the second DA session suggest that they were more able to identify correct definitions for the vocabulary items from more implicit mediation Such an improvement in learners’ mediated scores provides evidence for vocabulary development according to the sociocultural framework A Kruskal Wallis test was also used to further investigate the differences between the groups in the first and second DA session The analy-sis revealed statistically significant differences among the groups at both sessions: session 1, H (3) =

Table 3.Group comparisons for the non-dynamic post- and delayed post-test.