Teacher-Based Assessment for Foreign Language Pragmatics TRƯỜNG ĐẠI HỌC MỞ BÀI GIẢNG GIÁO TRÌNH

Teacher-Based Assessment for Foreign Language Pragmatics NORIKO ISHIHARA Hosei University Tokyo, Japan Despite the growing interest in teaching second language (L2) pragmatics, the issue of assessment of learners’ pragmatic skills, particularly in the context of the classroom, seems to be less prominently discussed, even though the assessment is an integral part of the instruction This qualitative case study aims to demonstrate an operationalization of a principle of pragmatics within the classroom context and demonstrate the effectiveness of teacher-based assessment of pragmatic competence grounded in Vygotsky’s sociocultural theory In freshman English courses at a Japanese university, the teacher researcher implemented pragmatics-focused instruction throughout a semester based on empirically established information on speech acts in English The instructor used collaboratively developed authentic assessment tools, such as reflective writing, rubrics, role-plays, and self/peer-assessment, and facilitated interaction and assessment in the learning process These assessments were designed (a) to elicit learners’ pragmalinguistic competence to use community norms, (b) to elicit their sociopragmatic awareness of the consequences of their own pragmatic language choice, and (c) to evaluate the extent of the match between learners’ intention (illocution) and interlocutors’ interpretation (perlocution) Examples of the assessments actually used in the classroom and learner language will be shown, and the instructor’s assessment will be interpreted in the sociocultural framework (Vygotsky, 1978) P ragmatic competence has long been identified as an essential component of communicative competence (e.g., Bachman, 1990; Canale & Swain, 1980; Hymes, 1972), and there has been a growing interest in addressing pragmatics in the second (L2) or foreign language (FL) curriculum as shown in the publication of pedagogically oriented articles (e.g., Crandall & Basturkmen, 2004; Eslami-Rasekh, 2005; Félix-Brasdefer, 2006; Ishihara, 2007; Martínez-Flor & Usó-Juan, 2006; Pearson, 2006; Washburn, 2001) and practical teachers’ resources (e.g., Bardovi-Harlig & Mahan-Taylor, 2003; Martínez-Flor & Usó-Juan, in press; Tatsuki, 2005; Tatsuki & Houck, in press) Since the 1990s, researchers have made vigorous efforts to develop and validate a battery of tests for learners’ pragmatic competence (e.g., Hudson, Detmer, & Brown, 1995; Liu, 2007; TESOL QUARTERLY Vol 43, No 3, September 2009 445 Röver, 2005) Validity and reliability of multiple methods for eliciting pragmatic comprehension and production (such as written and oral discourse completion tasks, multiple-choice tasks, role-play tasks, discourse self-assessment tasks, role-play self-assessment, and retrospective verbal report) have been investigated for both L2 and FL contexts (e.g., Brown, 2001; Cohen, 2004; Enochs & Yoshitake-Strain, 1999; Hartford & Bardovi-Harlig, 1992; Hinkel, 1997; Hudson, 2001; Kasper & Dahl, 1991; Rintell & Mitchell, 1989; Rose, 1994; Yamashita, 1996) These assessments are often carried out with trained raters under experimental conditions However, assessment of learners’ pragmatic skills in the context of the classroom seems to be less prominently discussed, even though assessment is an integral part of instruction From a teachers’ perspective, despite the proposed instruments developed through research, it has not been clear in what specific ways teachers can assess learner language, what feedback they should give to learners, and how they can use assessment for subsequent instruction conducive to further pragmatic development This study aims to demonstrate an operationalization of a principle of pragmatics within the classroom context and to demonstrate the effectiveness of teacher-based assessment of pragmatic competence grounded in Vygotsky’s sociocultural theory As the label suggests, teacherbased assessment (also termed classroom(-based) assessment, alternatives (in) assessment; Brown & Hudson, 1998; Norris, Brown, Hudson, & Yoshioka, 1998; Rea-Dickins, 2008; Shohamy, 1996) is conducted by the actual instructor (not an outside researcher or trained rater) and designed for facilitating learners’ development within the everyday classroom context Teacher-based assessment contrasts with traditional assessment, which often takes the form of standardized multiple-choice instruments and measures learners’ knowledge of language in numerical terms, often for the purpose of rank-ordering them (McNamara, 2001) CHALLENGES AND COMPLEXITIES OF ASSESSING OF PRAGMATICS Some of the challenges in teaching and assessing pragmatics stem from pragmatic variability in different community practices Due to macrosocial variation (Schneider & Barron, 2008) (e.g., regional, social, gender, ethnic, and generational differences in pragmatic norms), an appropriate or preferred range of norms of linguistic behavior manifests differently depending on the interlocutors’ individual personalities and social backgrounds (McNamara & Röver, 2006) At the same time, pragmatic language use is sensitive to various (and often subtle) contextual factors leading to microsocial variation (Schneider & Barron, 2008), 446 TESOL QUARTERLY (variation depending on, e.g., the interlocutors’ relative social status, psychological/social distance, and degree of imposition) Another layer of complexity comes into play for L2 pragmatics Because of their multicultural subjectivity, L2 speakers have been found to intentionally resist what they perceive as native speaker norms at times despite an awareness and linguistic command of such norms (e.g., Ishihara, 2006; LoCastro, 2003; Siegal, 1996) Learners’ pragmatic choices, whether an accommodation or a resistance to perceived community norms, are born out of their negotiation of subjectivity and exercise of agency Thus, care must be taken in instruction not to impose native speaker norms on learners’ production (e.g., Kasper & Rose, 2002; Thomas, 1983), and in assessment not to penalize learners for deliberate nontarget-like behavior To so could be interpreted as linguistic imposition or even cultural imperialism (e.g., Canagarajah, 1999; Kasper & Rose, 2002; Phillipson, 1992) In fact, a key to teacher-based assessment of pragmatics may be to use distinct evaluative strategies for learners’ receptive awareness and productive skills Teachers need to assess learners’ pragmatic comprehension based on the range of L2 community norms so that learners can eventually interpret community members’ intended meaning accordingly in social interaction At the same time, teachers must evaluate learners’ pragmatic use of language not in terms of how much it approximates native speaker norms but based on learners’ intended meanings and the nuances they choose to communicate, whether they elect to converge on or diverge from community norms Given that language proficiency (and in this case, pragmatic competence) is context-bound (McNamara & Röver, 2006), learners’ pragmatic language use would ideally be assessed in authentic social interaction, taking into account how they negotiate their message, identity, and cultural affiliation in that particular context TEACHER-BASED ASSESSMENT OF L2 PRAGMATIC COMPETENCE Considering these complexities, teacher-based assessment procedures appear to be well suited for L2 pragmatics Because teachers ultimately need to assess to what extent learners’ language production is likely to achieve their intended meaning in the social context, assessment appears to be inherently sensitive to students’ communicative intents The overall degree of success in communication must be investigated in terms of the ways in which learners intend to negotiate their identities through the use of the L2 Learners’ levels of pragmatic awareness can only be assessed by tapping into their pragmatic comprehension and metapragmatic analysis of the social context In teacher-based assessment, teachers can TEACHER-BASED ASSESSMENT FOR FOREIGN LANGUAGE PRAGMATICS 447 elicit learners’ intent and pragmatic awareness in collaborative dialogue and integrate it into day-to-day instruction and assessment on an ongoing basis Teacher-based assessment is often performance based in that learners produce written or spoken language in assessment as they perform authentic or simulated tasks In this direct form of assessment, learners draw on their prior knowledge and relevant skills, often in interactive discourse (Brown, 2004; O’Malley & Valdez-Pierce, 1996) The defining characteristics of teacher-based classroom assessment include (but are not limited to) the use of multiple and complementary instruments, an active role played by learners, the use of meaningful real-world tasks, the exercise of higher-order thinking, an emphasis on the process as well as the product, the integration of various language modalities, in-advance presentation of evaluative criteria to the learners (as well as other stakeholders), and the use of feedback as a way of supporting learning (Bailey, 1998; Brown, 2004; Brown & Hudson, 1998; Fox, 2008; Norris et al., 1998; O’Malley & Valdez-Pierce, 1996; Tedick, 2002) Because the goal of teacher-based assessment is greater student learning, that is, to enhance all learners’ development rather than just a selected few (Darling-Hammond, 1994; Lynch, 2001; Shohamy, 2001), learners’ competences are typically described or summarized in a descriptive narrative, written in terms of what learners are able to do, as well as what they need to work on Therefore, assessment provides diagnostic information regarding learners’ current state of knowledge and skills, assisting teachers in determining the subsequent course of instruction Validity and reliability of classroom-based assessment has been discussed among its proponents (e.g., for a detailed discussion, see Birenbaum, 1996; Brown & Hudson, 1998; Huerta-Macías, 1995; Lynch, 2001; Lynch & Shaw, 2005; McNamara & Röver, 2006; Moss, 1994) Although a certain degree of validity tends to be assumed based on the direct assessment and real-world (or simulated) nature of the tasks (e.g., Huerta-Macías, 1995), some have offered a cautionary note that validity should not be taken for granted and that there could be threats to (construct) validity (Brown & Hudson, 1998) Reliability of teacher-based assessment is often conceptualized differently from traditional standardized testing (Lynch & Shaw, 2005) Rater disagreements may result from differences in raters’ subjectivity, and in this case, the potential difference in rater reaction can be a natural consequence of pragmatic variability Also, reliability is called into question when a classroom instructor, who may not be professionally trained to be a rater, is the sole evaluator of students’ performance working in a time-constrained manner In addition, teacher-based assessment may be seen as relatively impractical, as instruments can be difficult to construct and more timeconsuming than traditional testing (Fox, 2008) It is important not to 448 TESOL QUARTERLY overestimate the credibility and strengths of teacher-based assessment Acknowledging these potential limitations and challenges can lead to further discussion about how the preliminary efforts for assessing L2 pragmatics presented in this article can be enhanced in future classroom contexts THEORETICAL UNDERPINNINGS OF TEACHER-BASED ASSESSMENT OF L2 PRAGMATICS It is important to stress that instruction and teacher-based assessment are part of a cyclic process and are inseparable from each other in facilitating learners’ language development.1 Theoretically consistent with this conceptualization of the role of assessment within the learning process is Vygotsky’s sociocultural framework (Fox, 2008; Rea-Dickins, 2008), particularly the concept of the zone of proximal development (ZPD), the notion that cognitive development occurs through language-mediated activity in interaction with others with more advanced cognitive ability (such as a teacher or a more capable peer) (Vygotsky, 1978) Closely related to the concept of ZPD is the notion of teachers’ (or peers’) scaffolding, which is typically mediated by language or cultural artifacts and consists of predetermined guidance or spontaneous feedback provided to learners based on observation of their competence (e.g., Lantolf, 2000; Lantolf & Thorne, 2006; Ohta, 2005; Wood, Bruner, & Ross, 1976) In teacher-based instruction and assessment of L2 pragmatics, scaffolding may be embedded in the instructional process, where teachers direct learners to the key contextual features in leading them to choose the appropriate language form Alternatively, teachers may point out language features that were not quite appropriate in students’ production to prompt self-reflection and improvement Teachers’ scaffolding is dialogically constructed to facilitate understanding and support cognitive development It may be individualized, based on each learner’s performance or targeted for a group (or subgroup) of learners in a classroom setting Scaffolding may also be provided by more competent peers Through successful other-regulated mediation, learners eventually appropriate and internalize the newly gained knowledge or skills and become self-regulated when they no longer need to rely on outside resources to carry out the task or access that awareness that has now become part of their cognitive repertoire (Vygotsky, 1978) The approach to assessment outlined in this article is in accordance with the principles of dynamic assessment (Ableeva, 2007; Lantolf & Poehner, 2004; Poehner, 2007), an approach associated with alternative assessment (see Fox, 2008) However, its methodologies for implementation seem unnecessarily restrictive in this particular FL classroom context TEACHER-BASED ASSESSMENT FOR FOREIGN LANGUAGE PRAGMATICS 449 THE CURRENT STUDY This article reports on a project conducted within the context of the classroom, where assessment was intertwined with instruction, facilitating learners’ pragmatic development and providing the teacher with diagnostic information about learners’ current state of knowledge and skills A case study approach was selected to examine and illustrate the complex and individualized process of pragmatic development or nondevelopment over the length of a course First, this article aims to demonstrate an operationalization of a principle of pragmatics within the classroom context by providing examples of assessment instruments for classroom use The study also intends to demonstrate ways in which learners’ pragmatic development (or nondevelopment) manifested itself to the classroom teacher within the constraints of the classroom The study was guided by the following research question: How can the development (or lack thereof) of learners’ pragmatic competence as assessed by a classroom teacher be characterized? Utilizing researchbased information, the pragmatics-focused instruction provided in the classroom centered on request discourses, but the pragmatics of giving commands, asking for permission, and making invitations were occasionally addressed within the oral skills component of the class METHODS This ethnographic case study reports on assessment efforts of a teacher in her university EFL classrooms in Japan The curriculum for freshman English centered on the development of oral skills as well as the use of English in sociocultural contexts through pragmatics-focused instruction The class met once a week for 90 minutes for 16 weeks, and approximately half of the class time was devoted to the instruction and assessment of FL pragmatics The instructional goal for the pragmatics component was to heighten learners’ pragmatic awareness in general, as well as to improve the contextual appropriateness of their use of requests in English discourse Participants Of the 71 first-year university students in three EFL intact classes, 58 consented to participate in this study They were 41 males and 17 females in their late teens majoring in business administration at a large university in Tokyo They were placed in these required English courses based on their TOEFL-ITP Level scores obtained at the start of the academic year On the paper-based TOEFL, students in two of the courses scored 370–399, whereas those in the other course scored 450 TESOL QUARTERLY 400–429 In the initial background survey, the participants reported having very little cross-cultural experience in the past,2 and only one spoke another language (heritage language Chinese) other than the required first and second FLs they were studying at the university The great majority of students indicated that they wished to improve their communicative skills in English in the course Assessment Instruments Integrated Into Instruction One of the goals of this article is to propose several pragmatics-focused assessment instruments for classroom use that can be further refined as a result of the discussion stimulated by this type of research The assessment instruments, which doubled as instructional tools, were designed by adapting the principles of classroom-based assessment of pragmatics recently proposed (Ishihara, in press), and earlier versions were collaboratively piloted and adapted in another FL context in which Japanese pragmatics was taught (Ishihara, Aoshima, & Akikawa, 2008) The instruction was conducted in English except for written first-language (L1) translations that accompanied some of the instructions and assessment criteria and the written reflections and request data that learners provided in their L1 Japanese Table shows the components of the instruction/assessment The instruction/assessment consisted of learners’ observation of L1 and L2 pragmatics (a, c, e), analysis of the context-language relationship in the L1 and L2 (c, d, e, h, j), learners’ written and oral output (b, f, g), teacher’s evaluation of learners’ production (i), self-assessment of pragmatic language use and community interpretation (g, h), and collaborative assessment with the instructor based on learners’ subjectivity ( j) (See Appendices A and B for sample scenarios used for eliciting learners’ production along with written discourse completion tasks [DCT] [b], and teacher–learner collaborated assessment of intention and interpretation [j]) One of the key assessments was learners’ selfevaluation (h), which was conducted 10 weeks into the instruction after learners reflected on and improved their initial pragmatic language use elicited during the second week of instruction (b) In this self-assessment, the learners, in the role of an English-speaker,3 engaged in evaluating Two reported having gone abroad on a school trip; one visited Australia for two weeks Another student indicated maintaining contact with an American exchange student she knew through a high school class Learners were asked to play a role of a presumably competent speaker of English in this task so that the teacher could assess learners’ pragmalinguistic command without running the risk of penalizing learners’ deliberate pragmatic resistance (see Ishihara, 2008, for further discussion) TEACHER-BASED ASSESSMENT FOR FOREIGN LANGUAGE PRAGMATICS 451 TABLE Instructional Sequence and Assessment Instruction/assessment Brief description and purpose of the instruments a Initial reactions to language use in context Learners’ observation of the use of Hong Kong English based on Rose’s (1999) episode Assessment of learners’ awareness of pragmatic variation and possibility of negative pragmatic transfer Written dialogues elicited from learners through multiplerejoinder DCT Assessment of learners’ pragmatic production Assignment for learners to collect naturally occurring request dialogues in L1 Japanese or L2 English Facilitation of learners’ noticing (Schmidt, 2001) of the language of request in the given context In-class instruction on request mitigators (adapted from Yates, 2003) and learners’ written reflections on learning pragmatics Assessment of learners’ pragmatic awareness about the context–language relationship and awareness of speaker’s intention and listener’s interpretation Learners’ analysis of the relationship between contextual factors and the language of request based on the learnercollected data (c) Assessment of learners’ noticing and understanding of the context–language relationship In-class instruction and quiz on request strategies (supportive moves) and key expressions Assessment of learners’ pragmalinguistic control Self-improvement of the previously completed multiplerejoinder DCT dialogues (b), and scaffolded and unscaffolded role-plays using the same scenarios Learners’ self-reflection on their own language production (g) with guiding prompts Assessment of learners’ pragmatic (1) awareness of directness, politeness, and formality in context; and (2) awareness of speaker’s intention and listener’s interpretation; and (3) pragmatic productive skills Teacher’s assessment of learner’s written dialogues produced without scaffolding (g) Assessment of (1) directness, politeness, and formality in context; (2) choice and use of supportive moves; and (3) overall pragmatics-focused comprehensibility Elicitation of learners’ requests and intention in making those requests Learners’ and teacher’s collaborative assessment of the match between the speaker’s intent and the listener’s most likely interpretation Assessment of learners’ pragmalinguistic control, awareness of community norms, and awareness of speaker’s intention and listener’s interpretation b Production of written request discourses c Learners’ data collection in authentic L1/L2 discourse d Learners’ reflections on language use in context e Learners’ analysis of context–language relationship f Pragmalinguistic development and assessment g Learners’ self-revising, role-playing, and refining request discourses h Learners’ self-evaluation of written request discourse i Teacher’s assessment of written request discourses j Teacher–learner collaborated assessment of intention–interpretation match their own pragmatic awareness and production The prompts in this assessment included: Given the context, how appropriate was Carla’s request in terms of overall directness, formality, politeness, and tone (e.g., intonation, gesture, eye contact)? 452 TESOL QUARTERLY a What part of Carla’s language demonstrates appropriate levels of directness, politeness, and formality? (*1) b What part of her language may need improvement considering the appropriate level of directness, politeness, and formality called for by this situation? (*1) What should she have said? Write the actual expressions you think she could have used (*2) Request Strategies [supportive moves]: a What requesting strategies did Carla use appropriately? Check the ones she used on the left column, and write out the expressions she used in the right of the chart (*1) [given in the student worksheet] b What other strategies could she have used? What might she say? (*2) Listener’s interpretation: How you think Carla’s request sounds to Professor Johnson, considering the situation? Check the one that most likely represents his reactions Then explain why you think that is the case (*1) — Prof Johnson would be willing to give her an extension because _ — He would give her an extension but may not be very happy with Carla’s language because — He may not give her an extension because _ *1: Assessment of analysis (excellent, good, needs more work) *2: Assessment of language (excellent, good, needs more work) This self assessment was followed by the instructor’s assessment (i) of learners’ written request dialogues produced after oral role-plays The following evaluative criteria were provided to learners in both English and in Japanese ahead of time: A Overall directness, politeness, and formality; tone (e.g., intonation, use of eye contact, gestures) if applicable: very appropriate, somewhat appropriate, less appropriate, inappropriate B Choice and use of requesting strategies (e.g., offering a reason for the request, getting a precommitment, checking availability, promising to compensate, showing consideration for the listener, expressing apology/thanks): very appropriate, somewhat appropriate, less appropriate, inappropriate C Overall comprehensibility of the speaker’s intention (in terms of appropriacy, not accuracy) (e.g., pronunciation, word choice, grammar, sequencing): highly comprehensible, somewhat comprehensible, less comprehensible, incomprehensible TEACHER-BASED ASSESSMENT FOR FOREIGN LANGUAGE PRAGMATICS 453 Data Collection and Analysis Assessment instruments listed in Table served as the vehicles for collecting data The researcher also served as the classroom instructor in the three intact EFL classes, so research was conducted through participant observation To ensure authenticity of the classroom data, student work was collected as part of the day-to-day instructional routine Instructional considerations were paramount precisely because the goal of the research was to investigate teacher-based assessment within the authentic environment and limitations of the classroom The data consisted of the instructor’s field notes taken while observing instruction/assessment, class documents (i.e., instructional activities and handouts, reflections, assessments, see Assessment Instruments section), and written questionnaires (i.e., initial student background survey, midterm reflective questionnaire, and course evaluations) For enhanced reliability, data were collected through multiple written sources over the spring semester of 2008 and triangulated for enhanced credibility (Patton, 2002) Ideally, learners’ online pragmatic competence would be assessed orally (as planned originally for this study) and in natural discourse if possible Because these EFL learners’ were extremely weak in listening and speaking, however, the first phase of pragmatics-focused instruction reported in this article relied on learners’ strength in writing as a projective measure of their speech Written language also facilitated teacher’s and learners’ pragmatic reflective analysis and assessment Because these participants almost never had an opportunity to use spoken English in authentic contexts, assessing simulated written dialogues was the most practical approach as in many other FL settings This is not to argue that assessment of pragmatics in oral language is not amenable to teacher-based assessment (see Ishihara et al., 2008, for oral assessment of pragmatics in a FL Japanese classroom) Recording and assessment of oral language in as much natural context as possible would be desirable in the subsequent course of instruction when learners’ oral fluency has improved and anxiety associated with spoken English has diminished To demonstrate learners’ progress in learning pragmatics (or lack thereof) as it manifested to the classroom teacher in the assessment, the data were analyzed deductively; the teacher researcher first sought evidence of noticeable pragmatic development or its absence To triangulate the data and the instruments, data from each assessment instrument and each learner were analyzed separately; then the constant comparative method (Merriam, 1998) was used to identify themes or tendencies that cut across the participants or assessments Through this cross-case analysis, similar perspectives and patterns of development or nondevelopment emerged as preliminary themes Subsequently, the 454 TESOL QUARTERLY All but one participant who submitted the written reflection mentioned that they learned a variety of request expressions in English that were associated with various contextual factors, demonstrating an awareness of microsocial variability The reflections revealed participants’ misconceptions about requests and politeness in English Their discoveries included noticing that English contains highly courteous expressions in English (which they thought were nonexistent) that are comparable to Japanese honorifics and a realization that they need to learn various English expressions for authentic communication Some learners felt that English is much more sensitive to context than Japanese as seen in the following quote:4 [I used to choose from various English expressions of request for no particular reason, but I learned that a variety of request expressions exist to suit a range of situations I learned that English expressions of request vary not only for the listener’s status but also for the magnitude of imposition I also learned what sort of language makes the request more polite While there is no ranking among Japanese honorifics, English has a ranking of politeness, which surprised me.] Awareness of the potential match and mismatch between a speaker’s intention and a listener’s interpretation is also shown in the great majority of learners’ reflections They also mentioned their learning of specific pragmalinguistic details, such as the level of politeness, directness, and formality of imperatives with please; the modals could, would, and may; and the functions of request mitigators However, awareness of macrosocial pragmatic variability was found in fewer learners’ reflections Several showed awareness of and interest in differences in pragmatic norms in World English varieties, such as American, Hong Kong, British, and Australian Englishes: 456 Most of the participants’ reflections were written in Japanese and their comments have been translated by the researcher and another bilingual teacher of Japanese and English using the back translation technique All participant names are pseudonyms TESOL QUARTERLY [ Just as we discussed [in class] about why Hong Kong English is [sounds] not very polite, I am curious about English expressions that vary according to the speakers in different countries and why they vary.] [I was amazed by the fact that the same English can be interpreted very differently in different countries I felt it is necessary to understand even direct expressions someone uses to me are just the way people speak English in that country.] Introduced to a much lesser extent in instruction, other macrosocial contextual factors, such as gender, ethnicity, generation, and socioeconomic status, were not mentioned in any of the learners’ reflections Another set of prompts facilitated analysis of the relationship between context and language (assessment e), based on the assessment (c) in which learners collected authentic request discourses in English or Japanese Learners analyzed how relative social status (S), distance (D), and imposition (I) influence the language of request and provided their hypotheses on how S, D, and I may influence the use of request strategies (i.e., supportive moves) These tasks come out of the learners-as-researchers approach (Tanaka, 1997; Tarone & Yule, 1989), in which learners are encouraged to observe and analyze pragmatics of the authentic use of English Because of the lack of instructional time, these bilingual prompts were given as an optional assignment, and the instructor provided narrative feedback Below is a sample of learner analysis and instructor feedback: [Great S/D/I make the request polite and small S/D/I make it less so In my observation, S and D are important and I is less influential, because I think in close relationships a request with a major imposition can be made less politely.] TEACHER-BASED ASSESSMENT FOR FOREIGN LANGUAGE PRAGMATICS 457 Instructor feedback: Yes, often true in Japanese! Excellent observation How about in English? [When S/D/I are great, I think what you say before and after, such as the way you give thanks, get a precommitment, and check on availability, also becomes more polite along with expressions of the request In contrast, when S/D/I are minor, the way you give gratitude, for example, becomes less polite and can be just “thanks.”] Instructor feedback: Very good analysis Are there any differences in the impact of S/D/I on the use of strategies between in English and in Japanese? This learner reflected on his Japanese data and offered an astute analysis that the magnitude of imposition tended to influence the politeness in Japanese to a less degree than relative social status or social/psychological distance, which is an observation similar to that in Hill, Ide, Ikuta, Kawasaki, and Ogino (1986) The teacher commended his insights and invited the learner to analyze pragmatics of English to which this tendency may not apply and to further refine his hypothesis Some of the assessments and instructional activities were designed to focus on learners’ pragmatic production During the second week of instruction, four written dialogues were elicited from learners through multiple-rejoinder DCT (Appendix A) for informal diagnosis (b) Six weeks later, learners revised their own initial responses for an assignment and were introduced to the criteria (i) to be used in assessing the assignment They were encouraged to review past handouts and notes, which included sample responses to the same DCTs made by pragmatically competent speakers of English Learners conducted role-plays in class based on their own revised dialogues as well as a few of their peers’ dialogues, role-played again without the written scaffolding, and wrote their best responses individually without referring to previously composed dialogues (g) Illustrations of pragmatic development are provided from two learners, Jun and Yu Differences in their pragmatic language use suggest different learning processes that became apparent to the instructor through the use of the instrument (i) The instructor’s field notes (Entry 13) show that she modeled a student asking a professor for a paper 458 TESOL QUARTERLY extension to these learners first working in a pair and later again to the entire class because she felt that it might benefit the rest of the class as well Later, the teacher’s assessment of the written dialogues revealed that Jun diversified his repertoire of supportive moves and effectively improved his pragmatic use of language whereas Yu’s improvement was more limited: I modeled: “I’m really sorry, but I have three other papers due the same week and don’t think I can finish them all I was wondering if you could give me an extension for two days because I would really like to a good job.” They [Jun and Yu] both sighed in awe I demonstrated the same response for the entire class So Jun and Yu heard my modeling twice To my pleasant surprise, Jun picked up on it pretty well! Assessing their written dialogues, I found that he used all the request strategies I demonstrated and effectively expanded his use of strategies A success! What did Yu on the other hand? His request itself wasn’t sufficiently indirect/polite, and he didn’t pick up on the strategies like Jun, even though he has good potential Is it his lack of motivation, lack of linguistic practice/effort, distrust in classroom instruction, a combination of all these, or? Jun’s demonstrated the following initial language use during the second week of instruction: Carla: Hello, I’m student at your class I want you to postpone [Prof Johnson: But you knew the deadline, didn’t you?] Carla: Yes The instructor did not respond to the learners’ initial production but returned it to the learners weeks later In his assigned second revision after instruction and mediation through (a)–(f), Jun was able to produce three different expressions of request, greatly and appropriately enhancing the level of politeness, directness, and formality: Carla: Excuse me My name is Carla I’m a student of your class I will not finish doing my course papers So will you postpone due date of a course papers Would it be right if I handed my course papers in next week [Prof Johnson: But you knew the deadline, didn’t you?] Carla: Yes But Could you possibly postpone the due date of a course papers? However, even after learning about a range of supportive moves (request strategies), he did not demonstrate their use except for providing a statement of a general reason (i.e., not completing the paper) In the third attempt after scaffolded and unscaffolded role-playing and instructor feedback in class, Jun was now able to provide an apology, a detailed reason, and a promise to compensate His expressions of TEACHER-BASED ASSESSMENT FOR FOREIGN LANGUAGE PRAGMATICS 459 request, as well as his word choice (i.e., get extension/extend) were also further refined Carla: Excuse me My name is Carla I’m a student of your class May I get some extention [sic] for the deadline of a course papers? [Prof Johnson: But you knew the deadline, didn’t you?] Carla: I know the deadline I’m sorry But, I have another three long course papers at same week However I want my best on all course papers So, could you possibly extend the due date of a course papers for a few days? Using the assessment criteria (i), Jun’s language use was evaluated by the instructor as “very appropriate” in overall directness, politeness, and formality (Criterion A); very appropriate in the choice and use of request strategies (B); and highly comprehensible (C) Because Jun’s pragmatic development was noticeable, the instructor recalled previous individual interaction with him in and outside of the class in her field notes (Entry 13) in which she characterized him as a motivated active learner planning to study abroad Although his active cognition did not take a visible shape in class, he did sometimes discuss the assignment with the instructor individually (Field Note Entry 12) Although he fared averagely in class and completed no extra credit assignments, the other assessments showed that he sometimes elected to write his reflections in English (a, c, e) and revised his reflections in response to the instructor’s written feedback (e) These oral and written interactions with the instructor may have provided additional mediation that supported his pragmatic development Because these behaviors were explicitly encouraged in the class, many students exploited the opportunity as Jun did Generally learners’ interest in the pragmatics component of the course was high, as demonstrated in the midterm reflective questionnaire, but a few learners were often less engaged in instruction This was the case for Yu, despite his fine oral skills (Field Note Entry 5) and his wish to become able to communicate in English (initial background survey) In informal interaction with the instructor after class, which occurred almost every week, Yu disclosed a belief that he needed to live abroad to be able to learn English (Field Notes Entry 10) This apparent distrust in formal instruction may have derived from his Japanese–Chinese bilingualism.5 Yu often made a minimal effort for class assignments In fact, he abandoned his initial response to the multi-rejoinder DCT (b) halfway through; he did not write a response for the particular scenario under investigation In his assigned second revision (g), he wrote 460 He identified himself as Chinese growing up speaking both Chinese and Japanese fluently in Japan TESOL QUARTERLY Carla: Excuse me I have a favor to ask of you Could you postphone [sic] report? [Prof Johnson: But you knew the deadline, didn’t you?] Carla: Yes But I was sick since last month So I couldn’t did my report Because Yu did not provide his initial response, it is not clear whether or how the instruction thus far may have influenced his pragmatic language use After role-play interactions and mediation provided by his peers and instructor, Yu’s response was as follows: Carla: Excuse me I have a favor of you Could you postphone report? [Prof Johnson: But you knew the deadline, didn’t you?] Carla: Yes But I was sick since last month You know your report are so many I want to a good job So could you postphone the deadline two days? In this response, he persisted with his original excuse, which he invented perhaps as a versatile face-saving strategy Other than the strategy of promising to compensate (“I want to a good job”), Yu did not demonstrate expanding his repertoire of supportive moves and request expressions Compared with Jun’s performance, Yu’s pragmatic development was minimal Based on the assessment criteria (i), the instructor evaluated Yu’s language use as “less appropriate” in overall directness, politeness, and formality (Criterion A); less appropriate in the choice and use of request strategies (B); and somewhat comprehensible (C) DISCUSSION The previous examples demonstrate a range of participants’ pragmatic development and provide a modest example of how pragmatic language use could be assessed in a classroom context The assessment was designed to assist learners’ development and be diagnostic; thus, feedback was largely conveyed through narratives, and learners were invited to refine their analyses and resubmit them, allowing for a type of dialogic interaction with the instructor Although some learners took advantage of the mediated interaction in the recursive cycle of instruction and assessment and demonstrated noticeable development, as Jun did, others did not show as much improvement and provided less insightful analyses Yu’s pragmatic language use reflected less successful learning than Jun’s, even when the same instructor mediation was available Jun exploited the instructor’s modeling and internalized it more fully, whereas Yu only minimally appropriated the mediation As a result, Jun was able to demonstrate self-regulated use of supportive moves immediately after TEACHER-BASED ASSESSMENT FOR FOREIGN LANGUAGE PRAGMATICS 461 the mediated interaction, whereas Yu did so only modestly Indeed, Jun’s high engagement in dialogic interaction with the teacher regarding the instructional content was in stark contrast with Yu’s disinterest or lack of trust in the value of classroom language learning Their apparent differences in demonstrated pragmatic competence may be explained by accumulated effects of mediational benefits or lack thereof over the past several weeks of instruction As Vygotsky (1978) points out, the process of internalization occurs in the wake of a series of developmental events Even though Yu was a more visible and sociable interactant with the instructor on the surface, his dialogue with the instructor was never about pragmatics being studied in this class but about language learning in general Linguistically speaking, these two learners were comparable at the start of instruction (Jun scored 420 and Yu 423 on TOEFL; both were in a more advanced class); in fact, Yu may have had slightly more advanced oral skills in English It was Jun, however, who was constructively building dialogue more relevant to the instructional content with the instructor through the oral and written channel (such as assessments a, c, d, f, and h, along with a few individual oral exchanges) With this groundwork laid, Jun appeared to be more prepared to internalize the instructor’s oral modeling when it became available On the other hand, learners’ overall linguistic competence may influence their potential level of pragmatic development The participants in this study were in three intact classrooms, one of which had students with a 30-point higher proficiency on TOEFL, as indicated earlier Generally speaking, learners in this class seemed to have demonstrated more engagement in class activities; provided more sophisticated reflective analyses in assessments (d), (e), and (h); and demonstrated more appropriate language use closer to their intended meanings as revealed in assessment (f) It is possible that their investment in English learning was greater, that they were more aware of the importance of pragmatic competence, and/or that their grammatical competence better supported their pragmatic learning However, this observation may be tinted by the instructor’s bias and must be interpreted with caution The participants’ demonstrated pragmatic competence needs to be more rigorously investigated with multiple raters using additional measures in order to further examine this tentative impression Limitations of the Present Research and Suggestions for Future Studies This research study was conducted within the usual instructional constraints and has a number of limitations As mentioned earlier, instructional considerations were given priority over efforts to enhance 462 TESOL QUARTERLY research validity and reliability consistent with the intention to investigate teacher-based assessment within the authentic context of the classroom Less direct forms of assessment, such as written DCT (not oral role-play) and DCT discourse (rather than naturally occurring dialogues, which probably were next to nonexistent) were selected because of the aforementioned limitations in learners’ linguistic proficiency and the learning context The instructor was the sole evaluator of learners’ pragmatic awareness and production, rather than multiple trained raters assessing learners with reasonable reliability Such limitations reflect realities of many FL learning contexts and help us determine the potentials and limits of assessment within the classroom context Furthermore, teacher mediation researched in this study included written scaffolding and the oral feedback reported through the instructor’s field notes No audio- or video-recorded oral interactions in the classroom were studied because the reviewing of oral discourse would have been unrealistic to most FL teachers instructing a large number of students In the Vygotskyan framework, interaction, where higher level cognitive development occurs, does include interaction through a written channel However, in future studies, more thorough investigations of teacher– student and peer oral interactions would reveal more about the process and extent of learners’ pragmatic development Pedagogical Implications Just as language learning involves participation in the L2 community and construction of learners’ L2-speaking identity (Pavlenko & Lantolf, 2000), instruction in pragmatics in the classroom, particularly in a FL context, can function as a scaffolded process of socialization into an imagined L2 community (Norton, 2001) Learning L2 pragmatics takes learners beyond a literal, surface level understanding of L2 utterances to interpret others’ intended cultural meaning and negotiate their own identity positioning in the social context of the L2 community Conventional thinking would suggest that norms and practices of the L2 community are typically learned through exposure to and participation in the target community (Lave & Wenger, 1991; Wenger, 1998) However, in the FL setting, learners’ direct contact with a local English-speaking community tends to be limited, and sometimes indirect exposure to the L2 culture can be encouraged only through media or formal instruction Through this limited exposure, learners may gradually construct imagined communities which then shape their learning, sense of self, and voice in those communities In a classroom setting, a situational approach, which employs simulated scenarios, can be used as a practical start in teaching pragmatics, along TEACHER-BASED ASSESSMENT FOR FOREIGN LANGUAGE PRAGMATICS 463 with learners’ occasional participation and observation of the target language community outside of the classroom Despite the criticism that the scenarios are not truly authentic for learners’ daily lives (Zuskin, 1993), a situational approach was selected to serve as a bridge, inviting learners to notice realistic reasons for speaking English and to imagine themselves as members of an English-speaking community They would then gain their own voice in that community and construct an imagined bilingual identity for themselves The use of learner-generated scenarios (McLean, 2005) and efforts to enhance the visual aspects of the scenarios can assist in enhanced authenticity, which is to be exploited in the subsequent course of instruction Ultimately, pragmatics instruction, as provided in the FL classroom in the current study, is an invitation to this cross-cultural journey of exploring one’s identity and agency within the L2 The teacher-based assessment of L2 pragmatic competence introduced in this article aims to support learners in becoming competent community members able to exercise their agency as they either participate in community practices or actively resist and revise community practices at their will in today’s global society (Garrett & Baquedano-López, 2002; Schieffelin & Ochs, 1986) The assessments in this study were originally intended to (a) develop and elicit learners’ pragmalinguistic competence to use community norms, (b) develop sociopragmatic awareness of the consequences of their own pragmatic language choice, and (c) evaluate the extent of the match between learners’ intention (either accommodation or resistance to community norms) and interlocutors’ most likely interpretation in the context Although the first two goals of assessment have been met, it was found that the third one, as demonstrated in the instrument ( j), was a much more time-consuming process than expected Because of the limited instructional time and the learning context for this study, it was impossible within the given time frame to ensure learners’ adequate pragmatic awareness and appropriate language production (which would be a prerequisite for learners’ participation in or resistance to using community norms; Ishihara, 2006) This point needs to be well considered in planning pragmatics-focused instruction, provided that it aims to foster the construction of learners’ voice in the L2 community CONCLUSION Thus far, assessment of L2 pragmatic competence has centered largely on the development of test batteries (e.g., Hudson, Detmer, & Brown, 1995; Liu, 2007; Röver, 2005) Although the use of these instruments in the classroom context is sometimes warned against (e.g., Hudson, 2001), there has also been growing attention paid to the development of assessment in the classroom context (e.g., Cohen, 2004; Röver, 2004) 464 TESOL QUARTERLY Much work needs to be done and numerous hurdles must be overcome before we establish reasonably practical and reliable teacher-based assessment in terms of both construction of instruments and teacher development Yet, L2 pragmatics has currently been widely established to be amenable to instruction (see Kasper & Rose, 2002) If, in fact, pragmatics is to be integrated into L2 curriculum, learners’ pragmatic competence must also be assessed formally or informally in the classroom context Although assessment of pragmatic competence within instructional contexts poses a number of challenges, there is potential in using teacher-based assessment to help learners develop their pragmatic awareness and productive skills to communicate as they wish in the L2 Teachers and researchers working in collaboration can further this important enterprise ACKNOWLEDGMENTS I wish to thank the guest editors of this special issue of TESOL Quarterly and the anonymous reviewers for their continual support and helpful comments I am indebted to Sachiko Aoshima and Kumiko Akikawa for collaboratively refining earlier versions of some of the assessments used in this study My special thanks go to Sherrie Carroll for her insightful suggestions on an earlier draft of this article THE AUTHOR Noriko Ishihara is associate professor in EFL at Hosei University in Tokyo, Japan, and leads language teachers’ professional development courses in the United States and in Japan Her research interests include pragmatics and identity, and language teacher development She has recently co-authored Teaching and Learning Pragmatics (with A.D Cohen) REFERENCES Ableeva, R (2007) Assessing listening for development In R Alanen & S Poyhonen (Eds.), Language in action: Vygotsky and Leontievian legacy today (pp 352–379) Newcastle upon Tyne, England: Cambridge Scholars Bachman, L F (1990) Fundamental considerations in language testing Oxford: Oxford University Press Bailey, K M (1998) Learning about language assessment Boston: Heinle & Heinle Bardovi-Harlig, K., & Mahan-Taylor, R (Eds.) (2003) Teaching pragmatics Washington DC: Office of English Programs, U.S Department of State Retrieved June 15, 2008, from http://exchanges.state.gov/education/engteaching/pragmatics.htm Birenbaum, M (1996) Assessment 2000: Towards a pluralistic approach to assessment In M Birenbaum & F J R C Dochy (Eds.), Alternatives in assessment of achievements, learning processes and prior knowledge (pp 3–29) Boston: Kluwer Academic TEACHER-BASED ASSESSMENT FOR FOREIGN LANGUAGE PRAGMATICS 465 Brown, H D (2004) Language assessment: Principles and classroom practices New York: Longman Brown, H D., & Hudson, T (1998) The alternatives in language assessment TESOL Quarterly, 32, 653–675 Brown, J D (2001) Pragmatics tests: Different purposes, different tests In K Rose & G Kasper (Eds.), Pragmatics in language teaching (pp 301–325) Cambridge: Cambridge University Press Canagarajah, A S (1999) Resisting linguistic imperialism in English teaching Oxford: Oxford University Press Canale, M., & Swain, M (1980) Theoretical bases of communicative approaches to second language teaching and testing Applied Linguistics, 1, 1–47 Cohen, A D (2004) Assessing speech acts in a second language In D Boxer & A D Cohen (Eds.), Studying speaking to inform second language learning (pp 302– 327) Clevedon, England: Multilingual Matters Crandall, E., & Basturkmen, H (2004) Evaluating pragmatics-focused materials ELT Journal, 58, 38–49 Darling-Hammond, L (1994) Performance-based assessment and educational equity Harvard Educational Review, 64, 5–30 Enochs, K., & Yoshitake-Strain, S (1999) Evaluating six measures of EFL learners’ pragmatic competence JALT Journal, 21, 29–50 Eslami-Rasekh, Z (2005) Raising the pragmatic awareness of language learners ELT Journal, 59, 199–208 Félix-Brasdefer, C (2006) Teaching the negotiation of multi-turn speech acts: Using conversation-analytic tools to teach pragmatics in the FL classroom In K BardoviHarlig, C Félix-Brasdefer, & A Omar (Eds.), Pragmatics and language learning, Volume 11 (pp 167–197) Honolulu: University of Hawai’i at Manoa, National Foreign Language Resource Center Fox, J (2008) Alternative assessment In E Shohamy & N H Hornberger (Eds.), Encyclopedia of language and education: Vol Language testing and assessment (2nd ed.; pp 97–108) New York: Springer Science+Business Media Garrett, P B., & Baquedano-López, P (2002) Language socialization: Reproduction and continuity, transformation and change Annual Review of Anthropology, 31, 339–361 Hartford, B S., & Bardovi-Harlig, K (1992) Experimental and observational data in the study of interlanguage pragmatics Pragmatics and Language Learning, 3, 33–52 Hill, B., Ide, S., Ikuta, S., Kawasaki, A., & Ogino, T (1986) Universals of linguistic politeness: Quantitative evidence from Japanese and American English Journal of Pragmatics, 10, 347–371 Hinkel, E (1997) Appropriateness of advice: DCT and multiple choice data Applied Linguistics, 18, 1–26 Hudson, T (2001) Indicators for pragmatic instruction: Some quantitative tools In K Rose & G Kasper (Eds.), Pragmatics and language teaching (pp 283–300) Cambridge: Cambridge University Press Hudson, T., Detmer, E., & Brown, J D (1995) Developing prototypic measures of crosscultural pragmatics (Technical Report No 7) Honolulu: University of Hawai’i, Second Language Teaching & Curriculum Center Huerta-Macías, A (1995) Alternative assessment: Responses to commonly asked questions TESOL Journal, 5, 8–11 Hymes, D (1972) On communicative competence In J Pride & J Holmes (Eds.), Sociolinguistics: Selected readings (pp 269–293) Harmondsworth, England: Penguin Ishihara, N (2006) Subjectivity, second/foreign language pragmatic use, and instruction: Evidence of accommodation and resistance Unpublished doctoral dissertation, University of Minnesota, Minneapolis 466 TESOL QUARTERLY Ishihara, N (2007) Web-based curriculum for pragmatics instruction in Japanese as a foreign language: An explicit awareness-raising approach Language Awareness, 16, 21–40 Ishihara, N (2008) Transforming community norms: Potentials of L2 speakers’ pragmatic resistance In M Hood (Ed.), Proceedings of the 2008 Temple University Japan Colloquium on Language Learning (pp 1–10) Tokyo: Temple University Japan Ishihara, N (in press) Assessing learners’ pragmatic ability in the classroom In D Tatsuki & N Houck (Eds.), Speech acts and beyond: New directions in pragmatics Alexandria, VA: Teachers of English to Speakers of Other Languages Ishihara, N., Aoshima, S., & Akikawa, K (2008, April) Assessing pragmatic competence: A study of authentic assessment for foreign language pragmatics Paper presented at the Annual Conference of American Association for Applied Linguistics, Washington, D.C Kasper, G., & Dahl, M (1991) Research methods in interlanguage pragmatics Studies in Second Language Acquisition, 13, 215–247 Kasper, G., & Rose, K R (2002) Pragmatic development in a second language Malden, MA: Blackwell Lantolf, J P (2000) Sociocultural theory and second language learning Oxford: Oxford University Press Lantolf, J P., & Poehner, M (2004) Dynamic assessment: Bringing the past into the future Journal of Applied Linguistics, 1, 49–74 Lantolf, J P., & Thorne, S L (2006) Sociocultural theory and the genesis of second language development Oxford: Oxford University Press Lave, J., & Wenger, E (1991) Situated learning: Legitimate peripheral participation Cambridge: Cambridge University Press Liu, J (2007) Developing a pragmatics test for Chinese EFL learners Language Testing, 24, 391–415 LoCastro, V (2003) An introduction to pragmatics: Social action for language teachers Ann Arbor: The University of Michigan Press Lynch, B K (2001) Rethinking assessment from a critical perspective Language Testing, 18, 351–372 Lynch, B., & Shaw, P (2005) Portfolios, power, and ethics TESOL Quarterly, 39, 263–297 Martínez-Flor, A., & Usó-Juan, E (2006) A comprehensive pedagogical framework to develop pragmatics in the foreign language classroom: The 6R approach Applied Language Learning, 16, 39–64 Available from http://www.dliflc.edu/archive/documents/all16two.pdf Martínez-Flor, A., & Usó-Juan, E (Eds.) (in press) Speech act performance: Theoretical groundings and methodological innovations Amsterdam: Benjamins McLean, T (2005) “Why no tip?” Student-generated DCTs in the ESL classroom In D Tatsuki (Ed.), Pragmatics in language learning, theory, and practice (pp 150–156) Tokyo: The Japan Association for Language Teaching Pragmatics Special Interest Group McNamara, T (2001) Language assessment as social practice: Challenges for research Language Testing, 18, 333–349 McNamara, T F., & Roever, C (2006) The social dimension of proficiency: How testable is it? In Language testing: The social dimension (pp 43–79) Malden, MA: Blackwell Merriam, S B (1998) Qualitative research and case study applications in education: Revised and expanded from case study research in education San Francisco: Jossey-Bass Moss, P (1994) Can there be validity without reliability? Educational Researcher, 23, 5–12 TEACHER-BASED ASSESSMENT FOR FOREIGN LANGUAGE PRAGMATICS 467 Norton, B (2001) Non-participation, imagined communities and the language classroom In M Breen (ed.), Learner contributions to language learning: New directions in research (pp 159–171) Harlow, England: Pearson Education Norris, J M., Brown, J D., Hudson, T., & Yoshioka, J (1998) Designing second language performance assessments Honolulu: University of Hawai’i Press Ohta, A S (2005) Interlanguage pragmatics in the zone of proximal development System, 33, 503–517 O’Malley, M., & Valdez Pierce, L (1996) Authentic assessment for English language learners White Plains, NY: Addison-Wesley Patton, M Q (2002) Qualitative research and evaluation methods (3rd ed.) Thousand Oaks, CA: Sage Pavlenko, A., & Lantolf, J P (2000) Second language learning as participation and the (re)construction of selves In A Pavlenko & J P Lantolf (Eds.), Sociocultural theory and second language learning (pp 155–177) Oxford: Oxford University Press Pearson, L (2006) Teaching pragmatics in Spanish L2 courses: What learners think? In K Bardovi-Harlig, C Félix-Brasdefer, & A Omar (Eds.), Pragmatics and language learning, Volume 11 (pp 109–134) Honolulu, HI: University of Hawai’i at Manoa, National Foreign Language Resource Center Phillipson, R (1992) Linguistic imperialism Oxford: Oxford University Press Poehner, M (2007) Beyond the test: L2 dynamic assessment and the transcendence of mediated learning Modern Language Journal, 91, 323–340 Rea-Dickins, P (2001) Mirror, mirror on the wall: Identifying processes of classroom assessment Language Testing, 18, 429–462 Rea-Dickins, P (2008) Classroom-based language assessment In E Shohamy & N H Hornberger (Eds.), Encyclopedia of language and education: Vol Language testing and assessment, 2nd ed (pp 257–271) New York: Springer Science+Business Media Rintell, E M., & Mitchell, C J (1989) Studying requests and apologies: An inquiry into method In S Blum-Kulka, J House, & G Kasper (Eds.), Cross-cultural pragmatics: Requests and apologies (pp 248–272) Norwood, NJ: Ablex Rose, K R (1994) On the validity of discourse completion tests in non-Western contexts Applied Linguistics, 15, 1–14 Rose, K R (1999) Teachers and students learning about requests in Hong Kong In E Hinkel (Ed.), Culture in second language teaching and learning (pp 167–180) Cambridge: University Press Röver, C (2004) Difficulty and practicality in tests of interlanguage pragmatics In D Boxer & A D Cohen (Eds.), Studying speaking to inform second language learning (pp 283–301) Clevedon, England: Multilingual Matters Röver, C (2005) Testing ESL pragmatics: Development and validation of a Web-based assessment battery Frankfurt: Peter Lang Schieffelin, B B., & Ochs, E (Eds.) (1986) Language socialization across cultures Cambridge: Cambridge University Press Schmidt, R (2001) Attention In P Robinson (Ed.), Cognition and second language instruction (pp 3–32) Cambridge: Cambridge University Press Schneider, K P., & Barron, A (Eds.) (2008) Variational pragmatics Amsterdam: Benjamins Shohamy, E (2001) Democratic assessment as an alternative Language Testing, 18, 373–391 Siegal, M (1996) The role of learner subjectivity in second language sociolinguistic competency: Western women learning Japanese Applied Linguistics, 17, 356–382 Tanaka, K (1997) Developing pragmatic competence: A learners-as-researchers approach TESOL Journal, 6(3), 14–18 Tarone, E., & Yule, G (1989) Focus on the language learner Oxford: Oxford University Press 468 TESOL QUARTERLY Tatsuki, D (Ed.) (2005) Pragmatics in language learning, theory, and practice Tokyo: The Japan Association for Language Teaching Pragmatics Special Interest Group Tatsuki, D., & Houck, N (Eds.) (in press) TESOL classroom practice series: Pragmatics volume Alexandria, VA: Teachers of English to Speakers of Other Languages Tedick, D J (Ed.) (2002) Proficiency-oriented language instruction and assessment: A curriculum handbook for teachers Minneapolis: University of Minnesota, the Center for Advanced Research on Language Acquisition Thomas, J (1983) Cross-cultural pragmatic failure Applied linguistics, 4, 91–109 Vygotsky, L S (1978) Mind in society: The development of higher psychological processes Cambridge: Harvard University Press Washburn, G N (2001) Using situational comedies for pragmatic language teaching and learning TESOL Journal, 10(4), 21–26 Wenger, E (1998) Communities of practice: Learning, meaning, and identities Cambridge: Cambridge University Press Wood, D., Bruner, J., & Ross, G (1976) The role of tutoring in problem solving Journal of Child Psychology and Psychiatry, and Allied Disciplines, 17, 89–100 Yamashita, S O (1996) Six measures of JSL pragmatics (Technical Report No 14) Honolulu: University of Hawai’i, Second Language Teaching & Curriculum Center Yates, L (2003) Softening short requests In K Bardovi-Harlig & R Mahan-Taylor (Eds.), Teaching pragmatics Washington DC: U.S Department of State, Office of English Programs Retrieved June 15, 2008, from http://exchanges.state.gov/ education/engteaching/pragmatics/ishihara.htm Zuskin, R D (1993) Assessing L2 sociolinguistic competence: In search of support from pragmatic theories In L F Bouton & Y Kachru (Eds.), Pragmatics and language learning, Monograph series: Vol Selected papers presented at the Annual Meeting of the International Conference on Pragmatics and Language Learning (6th, Urbana, IL, April 2–4, 1992) (pp 166–182) Urbana: University of Illinois at Urban Champaign APPENDIX A Sample Scenarios Used for Assessing Learners’ Production (b) 1) Chris and Pat are university students and roommates in an apartment near campus They are very close friends Last week, Chris was sick and missed two class sessions Since the exam is coming up soon, Chris needs to ask Pat to borrow her class notes Because Pat is a good student, Chris also wants to ask if Pat can study together and help with the test preparation Pat: Hey, how’s it going? Chris: Pat: Sure, no problem, I just need my notes back in a couple of days Chris: Pat: Okay Let’s plan on doing that then 2) John is in a large class taught by Professor Andersen She shared a very interesting book in class last week Since she offered to lend it to anyone who might be interested, John decides to go to her after class to borrow it This is his first time talking to Professor Andersen in private John: Prof Andersen: Oh, I’m sorry but I’ve already lent it out to someone else who was interested John: Prof Andersen: Sure, no problem TEACHER-BASED ASSESSMENT FOR FOREIGN LANGUAGE PRAGMATICS 469 APPENDIX B Teacher–Learner Collaborated Assessment of Intention and Interpretation ( j) Scenario: You are trying to some homework but your roommate Jenny is watching a sitcom on TV and has the volume up so loud that it is distracting you and making it hard to concentrate (a) Your best guess: Write an example of what most English speakers would say: (b) Your intention: Think of how you want to speak (1) I want to make the request in a way most preferred in the community (2) I would want my words to sound a little more (formal/informal), (polite/direct), or than normal but still within the range of acceptable behavior (3) I choose not to use common behavior because I want to communicate my intentions (or not communicate them at all) in my own way Specify what community norms you decide not to use and why you don’t want them: _ (4) Other (specify: ) (c) What would you say, if you decide to speak to her about this? (d) How does your roommate most likely interpret your request? She would be willing to turn the TV down because She would turn it down but may not be happy with your language because She may not turn it down because _ Teacher’s assessment Linguistic ability to use community norms (a) and (c) Awareness of most probable listener’s interpretation (d) Match between (b) learner goal/intention and most probable listener’s interpretation of (c) Overall Assessment Excellent Good Fair Needs more work Highly aware Aware Less aware Unaware Excellent Good Fair Needs more work Excellent Good Needs more work Teacher’s comments: 470 TESOL QUARTERLY