Corpus informed assessment of comprehension of conversational implicatures in l2 english

BRIEF REPORTS AND SUMMARIES TESOL Quarterly invites readers to submit short reports and updates on their work These summaries may address any areas of interest to Quarterly readers Edited by ALI SHEHADEH United Arab Emirates University JOHN LEVIS Iowa State University Corpus-Informed Assessment of Comprehension of Conversational Implicatures in L2 English NAOKO TAGUCHI Carnegie Mellon University Pittsburgh, Pennsylvania, United States & A number of studies have examined second language (L2) learners’ ability to recognize linguistic and contextual cues and use them to make inferences of speakers’ intentions conveyed indirectly (e.g., Bouton, 1992; Roăver, 2005; Taguchi, 2007) In these studies, learners accurate identification of implied intentions has been used to depict pragmatic development and factors affecting the facility and difficulty of comprehension Research has found that implicature type and conventionality affect L2 comprehension and its development Implicatures vary in their degree of strength; some implicatures are strongly conveyed, while others are weakly understood, owing to the number of linguistic and contextual cues to be processed (Sperber & Wilson, 1995) The greater the number of cues to be processed, the more extensive people’s search for meaning becomes, resulting in greater processing load In contrast, when implicatures contain conventional linguistic features or are embedded in predictable discourse patterns, extensive analysis of contextual cues becomes unnecessary, and comprehension becomes immediate and automatic (Sperber & Wilson) In accordance with these theoretical claims, L2 learners were found to be sensitive to the differential degrees of implicitness and layers of meaning Empirical evidence exists where learners’ comprehension showed distinct patterns across implicatures that involved different degrees of conventionality (e.g., Bouton, 1992; Roăver, 2005; Taguchi, TESOL QUARTERLY Vol 43, No 4, December 2009 Tesol Quarterly tesol205411.3d 30/12/09 20:04:25 The Charlesworth Group, Wakefield +44(0)1924 369598 - Rev 7.51n/W (Jan 20 2003) 739 2007) In Bouton’s study, 30 ESL learners took a written multiple-choice test with 33 short written dialogues that had various types of implicatures The learners’ comprehension of relevance-based implicatures became similar to native speakers’ after 4.5 years However, they still struggled with formulaic implicatures (e.g., sequence implicatures, Pope implicatures such as saying ‘‘Can fish swim?’’ to mean ‘‘yes.’’) Taguchi (2007), on the other hand, examined the comprehension of indirect refusals and opinions among Japanese EFL learners Indirect refusals were considered conventional because they followed a common, predictable discourse pattern (giving a reason for refusal) Indirect opinions were considered less conventional because they did not attach meaning to specific linguistic expressions or predictable patterns (e.g., indicating a negative opinion of a movie by saying ‘‘I was glad when it was over’’) These item types were incorporated into a listening test that contained a series of short dialogues followed by yes–no questions Learners’ comprehension was faster and more accurate for indirect refusals than for indirect opinions Learners made significant gains in both accuracy and comprehension speed (measured by response times) over weeks, with the degree of gain for accuracy outweighing response time These findings suggest that the comprehension of implicatures have developmental patterns When meaning is based on shared conventions, it is easier to comprehend because it allows the transfer of pragmatic knowledge from the first language (L1) This was the case in Bouton’s studies, where learners’ comprehension of relevance implicatures became native-like over time Similarly, Taguchi’s study found that the comprehension of refusals preceded that of opinions because the means to encode refusals were conventional and shared between L1 and L2 However, when the convention is culture specific, or not shared between L1 and L2, meaning becomes the most difficult to recognize This phenomenon was shown in Bouton’s studies wherein learners continued to have difficulty with Pope implicatures and in Taguchi’s study in the comprehension of nonconventional implicatures These studies used artificially created implicatures to examine the effect of conventionality across implicature types Garcia’s (2004) study is probably the only existing study that used purely naturalistic dialogues to examine pragmatic comprehension Using conversations from a corpus, she developed a multiple-choice listening test that measured nonnative English speakers’ ability to identify four types of speech acts: requests, suggestions, corrections, and offers Results found that there were significant differences between lower and higher proficiency participants in their identification of speech acts Follow-up linguistic analyses of individual items revealed four factors that hindered comprehension: unclear agent, disfluency features, unfamiliar use of modal verbs, and mismatch between context and speech acts 740 TESOL QUARTERLY Tesol Quarterly tesol205411.3d 30/12/09 20:04:26 The Charlesworth Group, Wakefield +44(0)1924 369598 - Rev 7.51n/W (Jan 20 2003) Garcia’s work showed that naturally occurring language data can inform the design of pragmatic comprehension measures Various implicatures used in the previous studies can be extracted from naturalistic dialogues and adapted to create an instrument that more closely reflects real life language use More effort is necessary in adapting corpora to develop assessment tools to systematically examine the hierarchy of difficulty and conventionality affecting pragmatic comprehension gleaned in the previous research Because previous studies showed that learners find it easier and faster to comprehend certain implicatures than others, and accuracy gains precede speed gains (e.g., Taguchi, 2007), this study examined whether these patterns are present when using a test that incorporates naturally occurring implicatures The study adapted implicatures from corpora to develop a listening test to measure pragmatic comprehension Using the instrument, this study examined whether or not accurate and speedy pragmatic comprehension develops over time across implicature types of different degrees of conventionality METHODOLOGY Participants Participants in this study were 48 Japanese EFL (English as a foreign language) learners in a college in Japan There were 16 males and 32 females (average age was 18.33 years) They averaged 6.14 years of formal English education in Japanese Their average TOEFL score was about 460 (SD 17.77) None had more than one month of experience living outside Japan In addition to the ESL learners, 25 native English speakers participated in the pilot study and provided baseline data Instrumentation The Corpora Two corpora of face-to-face conversations of American English were selected for this study, representing two registers: family/friends interactions and service encounter interactions Family/friends interactions were taken from the Santa Barbara Corpus of Spoken American English (SBC) (Du Bois, Chafe, Meyer, & Thompson, 2000), and the service encounter interactions were taken from the TOEFL 2000 Spoken and Written Academic Language Corpus (T2K-SWALC) (Biber, Conrad, Reppen, Byrd, & Helt, 2002) CORPUS-INFORMED ASSESSMENT OF COMPREHENSION OF CONVERSATIONAL IMPLICATURES IN L2 ENGLISH Tesol Quarterly tesol205411.3d 30/12/09 20:04:26 The Charlesworth Group, Wakefield +44(0)1924 369598 - Rev 7.51n/W (Jan 20 2003) 741 Development of Test Items This study examined EFL learners’ listening ability to comprehend more and less conventional implicatures More conventional implicatures convey conventional meaning and are invariable in a specific situation For instance, the expression ‘‘How can I help you?’’ represents a speech act of offer that takes place within the frame of shopping When one is familiar with conventions of the frame, the expression is understood as formulaic, and comprehension becomes automatic Another example is a refusal routine of giving an excuse for a refusal (e.g., saying ‘‘I’m busy’’ when refusing someone’s invitation to a party) These more conventional implicatures convey indirect meanings by fixed linguistic forms or through predictable discourse patterns Less conventional implicatures represent indirect meanings that are relatively less conventional and vary according to context, as in the following example: A: How was your presentation? B: It’s over, so it’s OK In this example, an open set of expressions is possible as B’s reply When expressing negative opinions of the presentation, options regarding how to express them are wide open (endless expressions of liking and disliking) Thus, compared with more conventional implicatures, less conventional implicatures are more idiosyncratic; they require more extensive inferential processing to derive meaning because the listener needs to process a greater number of contextual cues More and less conventional implicatures were sought for in the SBC and T2K-SWALC corpora Transcriptions of the corpora were read line-by-line, and the target implicatures were hand-coded based on the following criteria More conventional implicatures: (a) Indirect refusals: refusal responses to invitations, requests, and suggestions with a reason (e.g., saying ‘‘I’m busy’’ when refusing someone’s request for help) (b) Routines: fixed or semifixed expressions that commonly occur under certain situational conditions and functions (e.g., ‘‘It comes to $2’’ in a service encounter exchange and ‘‘That’s so sweet of you’’ when thanking someone) Less conventional implicatures: nonliteral comments or opinions that not involve conventional linguistic features or language use patterns (e.g., indicating a negative opinion of a dinner by saying ‘‘The dinner was late’’) 742 TESOL QUARTERLY Tesol Quarterly tesol205411.3d 30/12/09 20:04:26 The Charlesworth Group, Wakefield +44(0)1924 369598 - Rev 7.51n/W (Jan 20 2003) The implicatures were taken from the corpora if the conversational context was judged accessible to the beginner EFL learners whom this study targeted This initial process yielded 26 more conventional and 24 less conventional implicatures Then dialogues were adapted to better serve the level of the target learner group and goals of the research Because authentic conversation involved a long stretch of discourse, the conversation was shortened to reduce the effect of short-term memory Proper nouns or vocabulary items that were considered unfamiliar to the target EFL learners were replaced with familiar equivalents (e.g., ‘‘mud pie’’ changed to ‘‘apple pie’’) The dialogues were written as a conversation between a male and female speaker so that the test takers can easily distinguish the voice Pilot Test One For the first test, 24 dialogues with more conventional implicatures and 24 dialogues with less conventional implicatures were prepared Although the intended meaning of more conventional implicatures was relatively straightforward, concerning the less conventional implicatures, it was questionable whether the same intended interpretations arise across different people owing to their variable nature Hence, a pilot study was conducted to confirm the reliability of the interpretations An open-ended survey containing the 24 dialogues was prepared In each dialogue, the target indirect utterance was underlined Participants were instructed to supply the interpretation for the underlined nonliteral utterance as in the following example: A: Hey Nancy, happy birthday I want you to open this gift It’s from me B: Oh, thank you, Mike This is a pretty package When did you have a chance to wrap this? A: At school B: Look at this A Mickey Mouse watch That’s just what I always wanted The survey was administered to 10 native English speakers at a U.S university, and follow-up individual interviews were conducted For the majority of the items, they showed relatively uniform interpretation However, some items showed a mixed interpretation In the preceding example, five respondents interpreted the target utterance as positive (i.e., Speaker B likes the gift), while four people wrote the opposite (i.e., ‘‘Speaker B is not thrilled with the gift’’) One person indicated both Follow-up interviews revealed that this utterance could be taken as sarcasm and implies negative opinion about the gift Based on these results, three items that showed inconsistent interpretations were eliminated, resulting in a pool of 21 less conventional implicatures, from which target items were selected CORPUS-INFORMED ASSESSMENT OF COMPREHENSION OF CONVERSATIONAL IMPLICATURES IN L2 ENGLISH Tesol Quarterly tesol205411.3d 30/12/09 20:04:26 The Charlesworth Group, Wakefield +44(0)1924 369598 - Rev 7.51n/W (Jan 20 2003) 743 Pilot Test Two The first version of the listening test (hereafter pragmatic listening test or PLT) had a total of 42 items: practice items, 32 experimental items, and filler items that tested literal comprehension The experimental items had 16 more and less conventional implicatures Each item had a short dialogue followed by multiple-choice questions with four answer options (see Table 1) The participants listened to the dialogue and selected the statement that is correct based on the content of the conversation In the experimental dialogues, the correct answer was the speaker’s implied intention Correct answers are marked with an asterisk The PLT was computerized using the software Revolution (Runtime Revolution, 2003) and piloted with 25 native speakers of English at a U.S university Several items were revised, and the accuracy rate of the TABLE Sample Pragmatic Listening Test Items More conventional implicatures Refusal Mike: Nancy: Mike: Nancy: Hey Nancy, what are you doing? Do you wanna something tonight? I don’t know I was just gonna watch TV I wanna go out tonight Maybe we can go to the Japanese restaurant The new one just opened I don’t have any money this week to pay the bills *1 Nancy doesn’t want to go out tonight Nancy is going out tonight Nancy got a bill from the restaurant Nancy is watching TV with Mike Routine Salesclerk: Customer: Salesclerk: Customer: Salesclerk: Customer: Hi, how can I help you? Ah, could I get a small regular coffee, with milk? And a slice of apple pie For here or to go? To go, please Here’s a large cup, we don’t have small because we ran out of the small ones OK, thank you The man ordered cake *2 The man is taking the coffee out The man is having coffee in the shop The man ordered a large cup Less conventional implicatures Mike: Nancy: Mike: Nancy: Mike: Why are you working so hard, Nancy? Hey Mike My medical exam is coming up next week You know I took it before, but I have to get a better score to get into a medical school What was your score before? Ten fifty I’d be happy with seven hundred *1 Mike thinks Nancy’s exam score is good Mike thinks Nancy’s score is poor Nancy is a medical school student Nancy’s medical school exam is next month 744 TESOL QUARTERLY Tesol Quarterly tesol205411.3d 30/12/09 20:04:26 The Charlesworth Group, Wakefield +44(0)1924 369598 - Rev 7.51n/W (Jan 20 2003) revised items was confirmed with the same native speakers After administering the test to the EFL learners, internal consistency reliability was assessed Cronbach’s alpha was 0.92 for the full test, 0.83 for less conventional implicatures, and 0.84 for more conventional implicatures (0.76 for refusals and 0.72 for routines) Procedure and Administration The pragmatic listening test was administered individually to the EFL participants using Windows computers on campus at the beginning of the semester The participants were not informed about the purpose of the research The participants practiced two items and then proceeded to the 40 test items Immediately following each dialogue, a multiplechoice question with four answer options in English appeared on the screen Participants were told to read each answer option and choose the correct one by pressing the corresponding key from to Response time was measured from the moment the question appeared on the screen and to when the participant pressed the key The procedure was repeated at the end of the semester RESULTS Tables and display the descriptive statistics of accuracy scores and response times Not surprisingly, native speakers’ comprehension was almost perfectly accurate and much faster than EFL learners, and their TABLE Descriptive Statistics, Accuracy Scores Mean SD Min Max Time More conventional Refusals Routines Less conventional 8.35 4.31 4.04 6.98 2.84 1.74 1.75 2.31 3.00 0.00 0.00 2.00 14.00 7.00 8.00 12.00 Time More conventional Refusals Routines Less conventional 8.97 5.13 3.85 6.77 2.76 1.59 1.60 2.27 4.00 2.00 1.00 3.00 5.00 8.00 8.00 12.00 Native Speakers More conventional Refusals Routines Less conventional 15.29 7.88 7.42 14.67 0.75 0.34 0.72 0.96 14.00 7.00 6.00 12.00 16.00 8.00 8.00 16.00 EFL learners Note SD standard deviation Min minimum Max maximum CORPUS-INFORMED ASSESSMENT OF COMPREHENSION OF CONVERSATIONAL IMPLICATURES IN L2 ENGLISH Tesol Quarterly tesol205411.3d 30/12/09 20:04:26 The Charlesworth Group, Wakefield +44(0)1924 369598 - Rev 7.51n/W (Jan 20 2003) 745 TABLE Descriptive Statistics, Response Times Mean SD Min Max 14.32 13.39 14.86 16.98 5.11 4.81 6.02 5.96 6.39 7.18 4.46 9.35 31.17 26.37 35.96 43.77 12.12 11.63 13.05 15.60 2.46 3.22 3.26 5.10 6.74 5.89 5.67 7.87 17.79 23.16 22.04 33.56 7.61 6.67 8.63 8.33 1.32 1.46 2.00 2.40 5.14 4.40 4.09 5.15 10.14 10.48 12.28 16.32 EFL learners Time More conventional Refusals Routines Less conventional Time More conventional Refusals Routines Less conventional Native Speakers More conventional Refusals Routines Less conventional Note The time refers to the average number of seconds taken to answer each item correctly SD standard deviation Min minimum Max maximum response times were relatively uniform across implicature types For the EFL data (second test), the repeated-measures ANOVA revealed a significant effect of item type on comprehension accuracy, F 33.62 (p , 0.05) Significant differences were found between less conventional implicatures and refusals, as well as between refusals and routines, but not between routines and less conventional implicatures Concerning response times, the repeated-measures ANOVA revealed significant differences across item types, F 18.17 (p , 0.05), with significant differences in all contrasts Refusals took the shortest time to comprehend, followed by routines Less conventional implicatures took the longest time to comprehend Paired-sample t-tests were applied to compare accuracy scores and response times between the first and second tests for the three implicature types There was a significant gain in accuracy scores for the refusal items, t 3.50 (p , 0.05), but not for the other two item types: t 0.48 (n.s.) for less conventional implicatures and t 0.61 (n.s.) for routines Response times remained the same over time; however, response times for refusal items tended to became shorter over time (t 22.39, p , 0.05) DISCUSSION This study revealed a noticeable interaction between implicature type, comprehension difficulty, and development of pragmatic comprehension 746 TESOL QUARTERLY Tesol Quarterly tesol205411.3d 30/12/09 20:04:27 The Charlesworth Group, Wakefield +44(0)1924 369598 - Rev 7.51n/W (Jan 20 2003) Comprehension of indirect refusals was the easiest and fastest, and showed a profound development over a short period of months Because the discourse pattern (i.e., giving a reason for refusal) was shared between L1 Japanese and L2 English, learners were able to apply their L1-based convention to L2 comprehension and made further progress as their general listening ability improved over time In contrast, comprehension of routines showed very little improvement both in accuracy and response times, suggesting different conventionality for refusals and routines Compared with refusals, routines were found to have stronger association with specific situations or communicative functions For instance, the phrase ‘‘For here or to go?’’ occurs in a service encounter situation of buying food, and the meaning is fixed across similar situations The fixedness and invariant nature of routines typically assists our comprehension because they are processed as a chunk rather than as a series of isolated words and, therefore, meaning is immediately retrievable from long-term memory, as long as people are aware of the forms and their contextual requirements The low accuracy and little development observed in this study, however, suggest that the EFL learners who participated were unfamiliar with the target routine expressions Owing to the contextdependent, culture-specific nature of the routines, they were probably not able to pick them up naturally in a foreign language environment that is limited to the type of situations in which the routines occur Similar to routines, comprehension of less conventional implicatures were also found to be slow in developing As opposed to conventional implicatures, they were context independent and did not reflect fixed linguistic forms or customized discourse patterns of refusals They required the learner to infer the context of the conversation, beyond understanding the target implicature forms The findings suggest that the ability to comprehend less conventional implicatures does not develop as quickly as conventional ones owing to the extensive inferential bridge speakers use to arrive at correct interpretations Without constant symbolic representations, the comprehension of these implicatures relies on more bottom-up analyses of syntactic and lexical information and contextual cues Using these analyses, learners must understand the literal meaning of the expressions and then work deductively toward the speakers’ implied intentions Multiple levels of processing extend the degree of inferencing that learners must do, thereby increasing difficulty and slowing down processing speed These observations are supported by the comparison of comprehension speed between less conventional implicatures and routines Despite the same accuracy rate, comprehension speed was faster for the routines than for the less conventional implicatures Hence, although the linguistic conventions encoded in routines were unfamiliar and difficult CORPUS-INFORMED ASSESSMENT OF COMPREHENSION OF CONVERSATIONAL IMPLICATURES IN L2 ENGLISH Tesol Quarterly tesol205411.3d 30/12/09 20:04:27 The Charlesworth Group, Wakefield +44(0)1924 369598 - Rev 7.51n/W (Jan 20 2003) 747 to access for the EFL learners, once they understood the conventions, they were able to derive meaning quickly by taking advantage of the conventions Less conventional implicatures, on the other hand, did not reveal such facility effect in comprehension speed IMPLICATIONS FOR PRAGMATIC TESTING AND TEACHING Using artificial dialogues, previous studies identified different patterns of development between accuracy and comprehension speed Learners find it easier and faster to comprehend certain implicatures than others, and accuracy gains precede speed gains (e.g., Taguchi, 2007) This study examined whether these patterns are present when using a test that more closely reflects naturally occurring implicatures The corpus-informed test items used here largely confirmed the hierarchy of demands involved in pragmatic comprehension and its effect on development Hence, this study confirmed that the degree of indirectness and conventionality, as reflected in variation in accuracy and response times, is not merely a property of pragmatic theories It is ubiquitous in everyday conversation, and learners’ comprehension can be assessed as ability to comprehend a variety of implicatures, both conventional and nonconventional, which are present in real-life conversation The process of test development in this study suggests a number of issues for future research The most important one relates to the degree of modifications that must be made to the dialogues taken from the corpus to create plausible test items for the target beginner-level learners Owing to those modifications, the dialogues in the listening test were not truly authentic in that they were not presented as they occurred in actuality Although this study purposefully reduced the messiness of naturalistic discourse to control for extraneous variables, future research that targets advanced-level learners should explore ways to incorporate features of spoken discourse into assessment of pragmatic comprehension This study offers some implications for the teaching of pragmatics It revealed that comprehension of indirect refusals develops naturally in a short period of time without explicit instruction Hence, instruction could be more profitably targeted at item types that develop more slowly, namely, routines and less conventional implicatures Common situational routines in naturalistic conversations, films, and TV dramas can be brought to the class and taught by drawing students’ attention to situational variables such as setting, purpose of interaction, and speakers in the exchange In this way, students will be able to draw a clearer connection between routines and their situational requirements, and 748 TESOL QUARTERLY Tesol Quarterly tesol205411.3d 30/12/09 20:04:27 The Charlesworth Group, Wakefield +44(0)1924 369598 - Rev 7.51n/W (Jan 20 2003) retrieve the routines quickly when they encounter them in similar situations Functional routines can be taught by informing students that some linguistic forms serve multiple functions in context For instance, the form ‘‘I’m sorry’’ is used to apologize but also to show sympathy Teachers can provide a list of those multifunctional routines and supply different contexts that can accompany individual functions Then they can have students study a specific context and identify the correct interpretation of the function of the routines When teaching less conventional, idiosyncratic implicatures, the instructional focus should be placed on the overall analysis of linguistic and contextual features used to derive intended meaning Teachers could present dialogues from authentic sources with the target indirect utterance underlined and ask students what the speaker means by the utterance, as in the following example: A: Do you like your neighbors? B: We’re always visiting each other Question: Does Speaker B like his or her neighbors? When teaching this dialogue, teachers can explain that Speaker B does not provide a direct answer, but his or her reply meets the assumption of relevance at an underlying level Because we typically visit someone when we like the person, the most plausible interpretation would be that Speaker B likes his or her neighbors This step-by-step inferential process requires explanation and might take some time initially, but repeated practice in applying the process will gradually help students to develop inferencing skills at the level where the process becomes effortless and automatic Teachers can also encourage students to use contextual cues such as paralinguistic information (e.g., intonation), background knowledge, and experience to draw inferences These exercises will promote students’ awareness of indirect communication and a variety of linguistic and nonlinguistic resources that they can draw on to make proper inferences of the nonliteral meaning THE AUTHOR Naoko Taguchi is Assistant Professor of Japanese and Second Language Acquisition in the Modern Languages Department at Carnegie Mellon University, Pittsburgh, Pennsylvania, United Sates Her research interests include interlanguage pragmatics, second language fluency, and classroom-based research REFERENCES Biber, D., Conrad, S., Reppen, R., Byrd, P., & Helt, M (2002) Speaking and writing in the university: A multidimensional comparison TESOL Quarterly, 36, 9–48 CORPUS-INFORMED ASSESSMENT OF COMPREHENSION OF CONVERSATIONAL IMPLICATURES IN L2 ENGLISH Tesol Quarterly tesol205411.3d 30/12/09 20:04:27 The Charlesworth Group, Wakefield +44(0)1924 369598 - Rev 7.51n/W (Jan 20 2003) 749 Bouton, L (1992) The interpretation of implicature in English by NNS: Does it come automatically without being explicitly taught? In Pragmatics and language learning monograph series volume (pp 64–77) Ubana-Champaign: University of Illinois Du Bois, J W., Chafe, W L., Meyer, C., & Thompson, S A (2000) Santa Barbara corpus of spoken American English, Part Philadelphia: Linguistic Data Consortium Garcia, P (2004) Developmental differences in speech act recognition: A pragmatic awareness study Language Awareness, 13, 96115 Roăver, C (2005) Testing EFL pragmatics Frankfurt, Germany: Gunter Narr Runtime Revolution (2003) Revolution, (ver 3) [Software] Edinburgh, Scotland: Author Sperber, D., & Wilson, D (1995) Relevance: Communication and cognition (2nd ed) Cambridge: Cambridge University Press Taguchi, N (2007) Development of speed and accuracy in pragmatic comprehension in English as a foreign language TESOL Quarterly, 41, 313–338 750 TESOL QUARTERLY Tesol Quarterly tesol205411.3d 30/12/09 20:04:28 The Charlesworth Group, Wakefield +44(0)1924 369598 - Rev 7.51n/W (Jan 20 2003) ... conventions encoded in routines were unfamiliar and difficult CORPUS- INFORMED ASSESSMENT OF COMPREHENSION OF CONVERSATIONAL IMPLICATURES IN L2 ENGLISH Tesol Quarterly tesol205411.3d 30/12/09... (2002) Speaking and writing in the university: A multidimensional comparison TESOL Quarterly, 36, 9–48 CORPUS- INFORMED ASSESSMENT OF COMPREHENSION OF CONVERSATIONAL IMPLICATURES IN L2 ENGLISH Tesol... inconsistent interpretations were eliminated, resulting in a pool of 21 less conventional implicatures, from which target items were selected CORPUS- INFORMED ASSESSMENT OF COMPREHENSION OF CONVERSATIONAL