With the light that literature has shed on the merits of authentic videos, this paper aims to foreground two video input enhancement activities, namely annotating and captioning and argue that when embedded in authentic videos, annotations and captions aid EFL learners’ vocabulary acquisition and thus English listening comprehension.
DISCUSSION v HOW INPUT-ENHANCED AUTHENTIC VIDEOS SUPPORT ENGLISH LISTENING COMPREHENSION: A DISCUSSION FROM AN INTERACTIONIST PERSPECTIVE NGUYEN XUAN NGHIA* * Hanoi University of Science and Technology, nghia.nx89@gmail.com Received: 20/4/2019; Revised: 14/5/2019; Accepted: 17/5/2019 ABSTRACT With the light that literature has shed on the merits of authentic videos, this paper aims to foreground two video input enhancement activities, namely annotating and captioning and argue that when embedded in authentic videos, annotations and captions aid EFL learners’ vocabulary acquisition and thus English listening comprehension To this end, annotations and captions are discussed on the theoretical grounds of Multimodality and the Interactionist Theory of Second Language Acquisition (SLA) The paper concludes with implications for language teachers as to the use of input-enhanced authentic videos for educational purposes in the listening classroom Keywords: listening comprehension, authentic videos, input enhancement, multimodality, interactionist theory INTRODUCTION Technologies seem to have come to aid listening skill earlier and with greater diversity than the other three macro English skills Robin (2011) lists broadcasts, tape recorders, and talking pictures as some of the electronic mediators to the teaching of listening skill before 1975 Since the 1980s, these devices have been either substituted or made more sophisticated, or naturally both, by an unending source of authentic audiovisual materials, which in turn came along with the advent of the Internet and Web 2.0 However, as authentic materials were historically created by and intended for native speakers in contrast to those created for pedagogical purposes, they are highly challenging in terms of grammatical, lexical, genre, and cultural contents (Garrett, 2009) Authentic videos are thus dismissed as less appropriate for learners at levels other than intermediate or above (Guariento & Morley, 2001) One may consider this a valid account for language teachers’ attempts to intentionally reduce the difficulty of authentic materials, but as Taylor (1994) puts it, materials can only be truly authentic when language is simplified by no means With this understanding, the paper highlights annotation and caption as two video input enhancement techniques that help create pedagogic videos without altering their authenticity First concepts around authenticity and its benefits for listening comprehension are summarized Then literature is reviewed on two KHOA HỌC NGOẠI NGỮ QUÂN SỰ No 19 (5/2019) 105 v DISCUSSION theoretical tenets – multimodality and Interactionist SLA – on which the discussion of annotation and caption is based At the end, recommendations are made for language teachers in regards to the use of authentic videos for pedagogical purposes in their listening classroom AUTHENTICITY Though authenticity was soon a subject of ample discussion in the 1970s as a result of the Communicative Language Teaching approach, it has remained ambiguous in the field of applied linguistics It is the concern over the multifacetedness of authenticity – whether it is associated with the text per se, the interaction between the teacher and the learner, the tasks chosen, etc (Gilmore, 2007) – that has led scholars yet to address it fully For example, Wallace focuses on the material aspect and defines authentic materials as “real life texts, not written for pedagogic purposes” (1992, p.145), while Tomlinson accentuates the task, referring to an authentic task as “one which involves the learners in communicating to achieve an outcome, rather than to practice the language” (2013, p.19) Morrow’s (1977) definition is used in this paper to summarize those varying lens: “An authentic text is a stretch of real language, produced by a real speaker or writer for a real audience and designed to convey a real message of some sort” (p.13) Authentic materials can be classified in different ways On the grounds of modality, Genhard (1996) classifies authentic materials into three categories: authentic audiovisual materials, which integrate both pictures and sounds; authentic visual materials, which are image-based and wordless; and authentic printed materials, which are presented on paper Concerning mainly authenticity level of the text, Campos (1992) proposes three types: authentic text, adapted or simplified text, and creative text An authentic text is not simplified or modified to any degree, and represent language used by native speakers in genuine situations for the fulfilment of their social 106 KHOA HỌC NGOẠI NGỮ QUÂN SỰ No 19 (5/2019) needs An adapted or simplified text is characterized by the simplification of language for the purpose of grammar and vocabulary introduction and reinforcement And, a creative text is created on the basis of the sequence of contents covered in a textbook This view is shared by Geddes and White (1978), who align Campos’s (1992) paradigm to only two categories of discourse: unmodified authentic discourse and simulated authentic discourse, with the former being “authentic text” while the latter encompassing “adapted text” and “creative text” The diagram below represents types of authentic materials in light of the three mentioned approaches Figure 1: Types of authentic materials (synthesized from Genhard, 1996; Campos, 1992; and Geddes & White, 1978) Authentic materials have been widely proved beneficial for L2 listening comprehension in a few ways First, learners yield constant exposure to real-life situations that are unachievable inside the textbook-based classroom Scholars such as Gilmore (2004) highlight the cultural contextualization of learning as the unique value of authentic materials, particularly authentic videos Second, learners have access to incremental linguistic gains This is enabled by frequent repetitions of lexical, grammatical and phonological features made by native speakers in an unexaggerated fashion (Devitt, 1997), which in turn strengthens the learner’s cognitive load and thus communicative competences (Sweller & Chandler, 2007) Also, a body of research (e.g Melvin & Stout, 1987; Christopher & Ho, 1996) DISCUSSION v has converged that authentic materials produce an inspirational and motivational environment for listening In brief, the advantages of authentic materials provide an impetus for the replacement of purpose-written materials In the sections below, literature around two concepts foundational to the discussion in this paper – Multimodality and Interactionist SLA – is reviewed MULTIMODALITY Kress and Van Leeuwen define multimodality as “the use of several semiotic modes in the design of a semiotic product or event, together with the particular way in which those modes are combined – they may for instance reinforce each other, fulfil complementary roles or be hierarchically ordered” (2001, p.20) It is the number of representational modes involved in presenting a piece of information that differs multimodality from unimodality, i.e the use of one single representational mode (within the scope of this paper, bimodality, i.e the use of two representational modes, is viewed a form of multimodality) If the traditional listening classroom relies mostly on aural texts, the cohesion of aural and visual materials has been in vogue, allowed by an abundant source of videos These sorts of materials also mirror the complementary nature of multimodality in that one semiotic representation carries one set of meanings while another a different set (Guichon & Cohen, 2016) In the listening comprehension classroom, multimodality is a notion that is inseparable from audiovisual materials for the latter’s representational essence, so the benefits of videos are shared by multimodality In his 2014 study, Woottipong stated that language teachers showed a particular liking for videos since multiple modalities were more appealing than audio-only instructions This view is echoed in Oura (2001) who found that audiovisuals engaged, motivated, and captured the learners’ attention significantly better than unimodal texts Another set of benefits of multimodality concern paralinguistic nuances, that is how intonation, pitch, prosody, and other nonverbal expressions align with the meanings being made This is because learners are immersed into a whole range of communication situations from numerous contexts (Harmer, 2007) According to Jones and Plass (2002), learners who were exposed to aural and visual listening outperformed counterparts given audio-only listening, and these individuals also bore a lower cognitive load INTERACTIONIST LANGUAGE ACQUISITION SECOND Amongst an array of theories underlying SLA, e.g Universal Grammar Theories, Cognitive Theories, Sociocultural Theories, Interactionist Theory is of paramount importance Originally, interaction is described by Ellis (1999) as “the interpersonal activity that arises during face-to-face communication”, and also as “the intrapersonal activity involved in mental processing” (p.3) The former type of interaction refers to direct conversational exchanges between two interlocutors, in which the less competent one (usually the learner) requests for the more competent counterpart’s (usually the native speaker) acts of repeating, explaining, modifying or enhancing linguistic inputs whenever communication breakdowns occur (Gass & Selinker, 1994) The latter is essentially about the kind of cognitive activity within the learner’s mind that engages deep mental processing in the wake of episodes of meaning negotiation or input modification requests Research shows that making linguistic inputs (lexis and syntax) and paralinguistic inputs (stress and intonation) salient during conversational interactions is how the more proficient interlocutor not only keeps the conversation going in a direction that mutual understanding is attained (Pica, 1994) but also facilitates delayed developmental effects on the learner’s cognition (Gass, 1997) Within the domain of CALL, Chapelle (2005) has reshaped the concept of “interpersonal KHOA HỌC NGOẠI NGỮ QUÂN SỰ No 19 (5/2019) 107 v DISCUSSION interaction” by adding a “between person and computer” facet to the existing “between people” one She also notes three benefit groups of interaction as opportunities for negotiation of meaning, acquiring enhanced input, and focusing attention on linguistic form, and aligns them to three types of interaction as shown in the table below: Table 1: Alignment of types of interaction to their benefits (adapted from Chapelle, 2003) Types of interaction Interpersonal interaction Intrapersonal interaction Benefits Between people (learner – native speaker) Negotiating meaning Between person and computer (learner computer) Acquiring enhanced or modified input Within the person (learner)’s mind Focusing attention on linguistic form in the input How interaction with computer promotes L2 acquisition in light of Chapelle’s (2005) conceptualization is indexed in the learner’s exposure to enhanced input displayed on the computer screen During this process, the learner notices input forms and makes form-meaning mappings, then gradually internalizes those connections via a course of intrapersonal activity This standpoint by Chapelle provides thrust for a good deal of research For example, Borrás and Lafayette (1994) found that accompanying aural input with L2 subtitles in listening lessons offered learners input of two modes, to the latter which they referred to decode the former in terms of not only general messages but also linguistic forms and meanings In another study, Grace (1998) put learners in contact with L1 translations and annotations of different types (e.g written and visual) to aid comprehension of vocabulary, and concluded that those enhanced inputs remedied miscomprehension and prompted noticing This line of research has also been extended to the area of reading comprehension, with Plass et al (1998) providing multiple forms of annotations to help with vocabulary and proving their encouraging effects on learners’ acquisition These findings mirror 108 KHOA HỌC NGOẠI NGỮ QUÂN SỰ No 19 (5/2019) the need and also potential for the introduction of Interactionist SLA into instructional design Given Long’s (1991) argument about the nature of the Interactionist Theory that only when noticed and comprehended can input become valid, instructional materials or CALL tasks should be designed in a way that key linguistic features are made salient, and enhancement of linguistic input is offered (Chapelle, 2007) In the next parts, I provide definitions of the two input enhancement methods – annotation and caption, and discuss them on the grounds of Multimodality and Interactionist SLA to illuminate how they support listening comprehension ANNOTATION AND CAPTION Annotations are explanatory notes added to demystify the meaning of an unknown word and come in three fundamental forms: written, visual/ pictorial, and audiovisual (Jones, 2004) Written annotations can be L2 definitions or L1 translations of the target word Visual/pictorial annotations, the representation in imagery, and audiovisual annotations, the support with both pictures and sounds, may potentially overload listeners if embedded in an existing audiovisual layer and seem to be a more appropriate for multimedia reading texts, so are not within the scope of the current paper Captions refer to “on-screen text in a given language combined with a soundtrack in the same language” (Markham & Peter, 2003, p.332) It is important to distinguish captions from subtitles which refer to “on-screen text in the same language of the viewers that accompany the second language soundtrack of the video material” (ibid.) In alignment with the nature of authentic videos as previously mentioned, that is language input should not be simplified or modified, the term “captions” is employed to address the matter under examination in this study Guillory (1998) classifies captions into two types: keyword captions and full captions, with the former highlighting DISCUSSION v only words that are either targeted or tough in their meaning, pronunciation, or morphological structure by themselves, and the latter displaying the entire spoken text DISCUSSION 6.1 Annotated and captioned videos are multimodal One of the arguments considered in the present paper is that annotated and/or captioned videos are multimodal In fact, a typical video is multimodal for its aural and visual representations (with the understanding that bimodality is a form of multimodality as mentioned earlier) A written annotation, when added to a video, makes it threefold in modality, i.e aural, visual, and textual In a similar vein, when captions, be they keyword or full, are inserted into a video, they radically switch it from a piece of two-modality material into that of three Figures and below illustrate this It is also important to note that annotations and captions can appear synchronously in a same video, and language teachers can use different dynamics, depending on their specific aims For example, instead of presenting the textual content in form of either annotations or captions, they may so with both by providing keyword captioning of a lexical item on the bottom of the screen and its definition or translation in an annotation bubble on the top Figure Annotation of “the Celts” Integrating annotations and captions into videos clearly signifies the plurality of semiotic modes in accordance with Kress and Van Leeuwen’s (2001) definition of multimodal materials These modes consolidate one another and provide complementary and mutual support as in the examples above: the annotated definition of “the Celts” and the captioned keyword “immense” (the textual modality) synchronize with the speaker’s speech (the aural modality) This in turn generates an eminently interactive product that aids learning to a great extent as indicated by literature regarding multimodality, i.e linguistic gains, paralinguistic subtleties, and motivational factors (e.g Woottipong, 2014; Harmer, 2007; Jones & Plass, 2002) 6.2 Annotated and captioned videos promote listening comprehension from the perspective of Interactionist SLA The other argument made about annotated and captioned videos in this paper is that they offer comprehensible input to facilitate listening comprehension According to researchers (e.g Kim, 2015), L2 listening comprehension is a complex process, involving two processes: a topdown process and a bottom-up process In the topdown process, learners mobilize their “schemata or background knowledge” to gain the gist and main ideas of the aural text, while with bottom-up processing, they are drawn to discrete words and phrases to decipher the meaning and content (Kim, 2015, p.16) Figure Keyword captioning of “immense” KHOA HỌC NGOẠI NGỮ QUÂN SỰ No 19 (5/2019) 109 v DISCUSSION Regarding top-down processing, annotated videos serve well on this, in that the annotated content goes beyond explanation of a lexical word to the level that is directed at activation of background knowledge and scaffolding of comprehension To the best of my knowledge, this is lacking in previous studies, so purposely made prominent in this report For example, in a video comparing the size of the Sun with that of other stars in the universe, I created an annotation with the question “Is the Sun the largest star in the universe?” to elicit the learners’ prior knowledge about this matter and prepare them for what was coming next in the video (Figure 4) This conforms to Kim’s (2015) analysis of the top-down listening process that processing prior knowledge paves the way for learners’ “grasp of incoming information” (p.16) Bottom-up processing is catalyzed by interactions with both annotations of individual words/phrases and captions With annotations, the matter of concern is whether definitions or explanations should be provided in L1 as in Figure or L2 as in Figure In actuality, the speed of an authentic video would not afford listeners time and space to read L2 definitions, so L1 translations seem to be the ideal option This can be elucidated by a body of research about the inextricable link between L2 words, L1 translations, and pictures (e.g Paivio, 1986) Take a further look into Paivio’s dual coding theory for an example The theory postulates that there are two information channels in the learners’ mind: an L1 verbal channel and Figure 4: Annotation of a question 110 KHOA HỌC NGOẠI NGỮ QUÂN SỰ No 19 (5/2019) a nonverbal/imagery channel The L2 verbal channel is formed and developed owing to the fact that the L2 word is connected to the image and L1 verbal subsystem existing in the learner’s mind This can be further illustrated from Figure 5, that is by the time listeners hear the word “the Celts” and the image depicting these tribes pop up on the screen, the L1 translation “người xen-tơ” allows them to assimilate “the Celts” with its meaning thanks to their consciousness of the L1 and visual cues Captions provide half of this route, in this regard, as far as keyword captions are concerned This means that when hearing and seeing the captioned word and the corresponding image on the screen, listeners make an interconnection and thus the form-meaning mapping In fact, literature has proved that watching videos with keyword captions made it easier for learners to understand the content for decreased cognitive load which would otherwise be heavy under full caption condition (Guillory, 1998) Above all, the described forms of annotations and captions result in interactive experiences between the learners and the computer rather than between the learners and native speakers as originally stipulated by Interactionist theory of SLA Put it differently, they constitute a technological platform for the learners to interact with enhanced inputs and benefit from them for lexical acquisition and comprehension of aural texts (Chapelle, 2003) Figure 5: Annotation of “the Celts” DISCUSSION v CONCLUSION AND IMPLICATIONS The departure point for this paper is that authentic videos are beneficial for second language learners in listening comprehension, fundamentally thanks to genuine situations and linguistic and paralinguistic elements they provide On this premise, the paper revisits annotation and caption as two input enhancement activities that can be performed on authentic videos, and argues that they further aid learners’ linguistic gains and thus listening comprehension abilities From its discussion grounded on the theories of Multimodality and Interactionist SLA, the paper arrives at the following practical implications for language teachers who intend to use authentic videos in their listening classroom [1] Videos should be opted for with the perception that their authenticity is not reduced on any level [2] Annotations should expand from explanations and/or definitions of individual words/ phrases to posing of questions for prior knowledge activation and comprehension scaffolding as well When annotations are intentionally aimed at individual lexical items, they should be done so in L1 as opposed to L2 [3] Captions should be supplied in key words rather than in full for optimal learning experiences Perhaps, an experimental study is needed in the future to genuinely test the effect of these suggestions./ References: Borrás, I., & Lafayette, R C (1994) Effects of multimedia courseware subtitling on the speaking performance of college students of French Modern Language Journal, 78(1), 61-75 Campos, R O (1992) Authenticity in listening and written texts LETRAS, 1(25), 169-194 Chapelle, C A (2003) English language learning and technology: Lectures on applied linguistics in the age of information and communication technology Amsterdam: John Benjamins Publishing Chapelle, C A (2007) Technology and second language acquisition Annual Review of Applied Linguistics, 27, 98114 Chapelle, C A (2009) The relationship between second language acquisition theory and computer-assisted language learning The Modern Language Journal, 93, 741-753 Chapelle, C.A (2005) CALICO at center stage: Our emerging rights and responsibilities. CALICO Journal, 23(1), 5-16 Christopher, E., & Ho, S (1996) Lights, camera, action: exploring and exploiting films in self-access learning Taking Control: Autonomy in Language Learning, 185-200 Devitt, S (1997) Interacting with authentic texts: Multilayered processes The Modern Language Journal, 81(4), 457-469 Ellis, R (1999) Learning a second language through interaction Amsterdam: John Benjamins Garrett, N (2009) Computer-assisted language learning trends and issues revisited: Integrating innovation The Modern Language Journal, 93, 719-740 Gass, S, M (1997) Input, interaction, and the second language learner Hillsdale, NJ: Erlbaum Gass, S M., & Selinker, L (1994) Second language acquisition: An introductory course US: Lawrence Erlbaum Associates Geddes, M., & White, R (1978) The use of semi-scripted simulated authentic speech in listening comprehension Audiovisual Language Journal, 16(3), 137-145 Genhard, J., G (1996) Teaching English as a foreign language: A teacher self-development and methodology Ann Abor: The university of Michigan Press Gilmore, A (2004) A comparison of textbook and authentic interactions ELT Journal, 58(4), 363-374 Grace, C (1998) Retention of word meanings inferred from context and sentence-level translations: Implications for the design of beginning-level CALL software The Modern Language Journal, 82(4), 533-544 Guariento W., & Morley, J (2001) Text and task authenticity in the EFL classroom ELT Journal, 55(4), 347-353 Guichon, N., & Cohen, C (2016) Multimodality and CALL In F Farr, & L Murray (Eds.), The Routledge Handbook of Language Learning and Technology (509-521) Routledge Guichon, N., & McLornan, S (2008) The effects of multimodality on L2 learners: Implications for CALL resource design System, 36(1), 85-93 Harmer, J (2007) The practice of English language teaching UK: Pearson Education. KHOA HỌC NGOẠI NGỮ QUÂN SỰ No 19 (5/2019) 111 v DISCUSSION Jones, L (2004) Testing L2 vocabulary recognition and recall using pictorial and written test items Language Learning & Technology, 8(3), 122-143. Jones, L C., & Plass, J L (2002) Supporting listening comprehension and vocabulary acquisition in French with multimedia annotations The Modern Language Journal, 86(4), 546-561 Kim H S (2015) Using authentic videos to improve EFL students’ listening comprehension International Journal of Contents, 11(4), 15-24 Kress, G., & Van Leeuwen, T (20015) Multimodal discourse: The modes and media of contemporary communication London: Oxford University Press Long, M (1991) Focus on Form: A Design Feature in Language Teaching Methodology In K De Bot, R Ginsberg, & C Kramsch (Eds.), Foreign Language Research in CrossCultural Perspectives (39-52) Amsterdam: John Benjamins Markham, P., & Peter, L (2003) The influence of English language and Spanish language captions on foreign language listening/reading comprehension Journal of Educational Technology Systems, 31(3), 331-341 Melvin, B S., & Stout, D S (1987) Motivating language learners through authentic materials In W Rivers (Ed.), Interactive Language Teaching (44-56) New York: Cambridge University Press Paivio, A (1971) Imagery and verbal processes New York: Holt, Rinehart and Winston. Pica, T (1994) Research on negotiation: What does it reveal about second language learning conditions, processes, and outcomes? Language Learning, 44(3), 1-35 Plass, J L., Chun, D., Mayer, R., & Leutner, D (1998) Supporting visual and verbal learning preferences in a second-language multimedia learning environment Journal of Educational Psychology, 90(1), 25-36 Robin, R (2011) Listening comprehension in the age of Web 2.0 In N Arnold & L Ducate (Eds.), Present and Future Promises of CALL: From Theory and Research to New Directions in Language Teaching (93-108) CALICO Sweller, J., & Chandler, P A (2007) The effect of written text on comprehension of spoken English as a foreign language American Journal of Psychology, 120(3), 237-261 Taylor, D (1994) Inauthentic authenticity or authentic authenticity? TESL-EJ, 1(2) Retrieved from http://wwwwriting.berkeley.edu/TESL-EJ/ej02/a.1.html Tomlinson, B (2013) Introduction: Are materials developing? In B Tomlinson (Ed.), Developing Materials for Language Teaching (2nd ed.) London: Bloomsbury Academic Wallace, C (1992) Reading Oxford, UK: Oxford University Press Morrow, K (1977) Authentic texts and ESP In S Holden (Ed.), English for Specific Purposes (13-17) London: Modern English Publications Winke, P., Gass, S., & Sydorenko, T (2010) The effects of captioning videos used for foreign language listening activities Language Learning & Technology, 14(1), 65-86 Oura, G K (2001) Authentic task-based materials: Bringing the real world into the classroom Sophia Junior College Faculty Bulletin, 21, 65-84 Woottipong, K (2014) Effect of Using Video Materials in the Teaching of Listening Skills for University Students International Journal of Linguistics, 6(4), 200-212 SỬ DỤNG VIDEO ĐỜI THỰC CĨ CHÚ THÍCH, PHỤ ĐỀ ĐỂ NÂNG CAO KỸ NĂNG NGHE HIỂU TIẾNG ANH: THẢO LUẬN TỪ GÓC ĐỘ CỦA LÝ THUYẾT TƯƠNG TÁC NGUYỄN XUÂN NGHĨA Tóm tắt: Với việc nghiên cứu trước lợi ích video đời thực kỹ nghe hiểu Tiếng Anh, viết chứng minh thích phụ đề đưa vào video đời thực giúp người học cải thiện vốn từ vựng kỹ nghe hiểu Bài viết thảo luận tác dụng hai hình thức sở lý thuyết Đa phương thức Thuyết tương tác Lý thuyết thụ đắc ngôn ngữ thứ hai Ở phần cuối số gợi ý cho giáo viên sử dụng video đời thực có thích phụ đề vào mục đích giảng dạy Từ khố: nghe hiểu, video đời thực, thích, phụ đề, đa phương thức, lý thuyết tương tác Ngày nhận bài: 20/4/2019; ngày sửa chữa: 14/5/2019; ngày duyệt đăng: 17/5/2019 112 KHOA HỌC NGOẠI NGỮ QUÂN SỰ No 19 (5/2019) ... pictorial, and audiovisual (Jones, 2004) Written annotations can be L2 definitions or L1 translations of the target word Visual/pictorial annotations, the representation in imagery, and audiovisual... paper is that annotated and/or captioned videos are multimodal In fact, a typical video is multimodal for its aural and visual representations (with the understanding that bimodality is a form of... traditional listening classroom relies mostly on aural texts, the cohesion of aural and visual materials has been in vogue, allowed by an abundant source of videos These sorts of materials also