Teacher based assessment of l2 japanese

Ishihara, N (in press) DRAFT Teacher-based assessment of L2 Japanese pragmatics: Classroom applications In S Ross & G Kasper (Eds.), Assessing second language pragmatics Basingstoke, UK: Palgrave Macmillan Teacher-based assessment of L2 Japanese pragmatics: Classroom applications Noriko Ishihara, Hosei University, Columbian University Teachers College Japan Campus Introduction In this chapter, the potential utility and limitations of teacher-based assessment are explored in the Japanese-as-a-foreign-language classroom context Teacher-based assessment constitutes “a more teacher-mediated, context-based, classroom-embedded assessment practice,” which is situated in opposition to traditional formal assessment that is often externally set and administered (Davison & Leung, 2009, p 395) Teacher-based assessment is sometimes termed alternative (in) assessment, classroom(-based) assessment, or authentic assessment (e.g., Brown & Hudson, 1998; O’Malley & Valdes-Pierce, 1996; Rea-Dickins, 2008) Despite the rigorous efforts to measure learners’ pragmatic competence (e.g., Ahn, 2005; Brown, 2001; Enochs & Yoshitake-Strain, 1999; Hartford & Bardovi-Harlig, 1992; Hudson 2001; Hudson, Detmer, & Brown, 1992, 1995; Itomitsu, 2009; Liu, 2006; Rintell & Mitchell, 1989; Rose, 1994; Roever, 2005; Yamashita, 1996), their application to everyday classrooms long remained underdeveloped (Hudson, 2001) The assessment of learners’ pragmatic competence in classroom contexts has only begun to be explored recently even though assessment is an integral part of instruction From a teachers’ perspective, we need to know how to implement effective assessment for L2 pragmatics in the classroom; the same concern is true for researchers and teacher educators if pragmatics is to be promoted in L2 instruction and teacher development 155 I argue that the teacher-based assessment demonstrated in this chapter aims to be authentic in principle (albeit not necessarily in reality) in a twofold manner Firstly, teacherbased assessment aims to be reflective of the pragmatic variation that exists in authentic interactions at multiple levels Secondly, the assessment is made not only of learners’ pragmatic competence but also for its further development in the authentic classroom context (Cumming, 2009; Davison & Leung, 2009; Leung, 2007) I will detail each case in what follows 1.1 Assessing for research vs assessing outside of the laboratory Although pragmatic norms in the target language are used as a reference point in both instruction and assessment, they can differ depending on interlocutors’ subjectivity and social backgrounds as well as various (and often subtle) contextual factors and diverse community practices (e.g., interlocutors’ regional, social, gender, ethnic, or generational backgrounds, relative social status, psychological and social distance, interactional structures and sequences, Schneider & Barron, 2008) These multiple layers of pragmatic variation mean that teachers and learners have multiple moving targets that are both amorphous and elusive, which makes pragmatics-focused instruction and assessment all the more challenging When learners’ pragmatic performance is assessed by raters as part of the research or test method, a further complication is that pragmatic variation extends not only to the participants but to the raters as well Interrater reliability is crucial in studies in which learners’ pragmatic competence is measured with a single yardstick shared among the raters for each measure While we need to ensure through rater training that raters share the same (or a sufficiently close) understanding of the assessment criteria, it is possible that even with agreed-upon criteria, raters may make varied assessments by relying on their own range of pragmatic norms In such a case, 156 different ratings can be seen as a natural consequence of pragmatic variation Roever (2008) acknowledges that raters will have disagreements if they are to assess the politeness and appropriateness of learners’ pragmatic competence In fact, in Taguchi’s (2011) research, even with an initial norming session in which a five-point rating scale was provided with examples, the four raters, who varied in their backgrounds, took issue with different aspects of the learners’ production and came up with different ratings (e.g., verbosity, clarity of the message, level of directness, use of positive politeness strategies) Taguchi’s post hoc interview data show that the raters also weighed the criteria differently, which contributed to the weak interrater correlations In the second norming session, the raters formulated their perceptions of appropriateness and politeness and discussed their discrepancies to establish common ground in their rating criteria Taguchi concludes that despite the rater variation, overall interrater reliability was acceptable and that raters with diverse cultural backgrounds and teaching experiences may be normed and trained Thus, in research contexts that require standardization, a balancing act may be achievable that ensures reliable ratings that encompass some measure of rater variation In authentic contexts, the learners’ interactants are their real language appraisers, and they may not necessarily share a single yardstick They are likely to assess learners’ language use from a range of subjective perspectives, and they usually will not undergo rater training or norming This means that they may have different sets of assessment criteria as to what constitutes appropriate behavior when interpreting learners’ pragmatic language use One rater may notice pragmatic divergence in a learner and take offense, while another may be unconcerned For example, learners’ interlocutors may or may not interpret learners’ pragmatic behavior based on what they perceive as a range of native-speaker norms (see Iino, 1996; Ishihara & Tarone, 2009) They may assess leniently based on the perceived level of learners’ 157 pragmatic or linguistic competence, or they may have different expectations based on their perception of learners’ linguistic and cultural backgrounds Furthermore, because learners’ pragmatic use of language is shaped by their subjective perspectives, the range of the pragmatic norms they choose to utilize in their production may diverge from that of other target language speakers (Ishihara & Tarone, 2009; Siegal, 1996; Taguchi, 2012), which adds another layer of pragmatic variation In such a case, care must be taken in assessment so as not to penalize learners for non-target-like behavior for which they deliberately opted While learners’ pragmatic awareness or comprehension can be assessed in reference to what is typically viewed as appropriate language use in the target language, their production may be assessed appropriately based on the meaning and nuances they intend to convey, rather than on the range of target language norms (Ishihara, 2009) So if assessment of learners’ pragmatic competence is to be made more authentically in the field than in the laboratory, those layers of pragmatic variation that reside in the learners and their interactants must be taken into consideration 1.2 Teacher-based assessment in the classroom context Teacher-based assessment as explored in this chapter addresses teachers’ and learners’ needs in the classroom context Teacher-based assessment is often formative and integrated into instruction It can be performance-based in that learners produce written or spoken language available for assessment as they perform authentic or simulated tasks In this direct form of assessment, learners draw on their prior knowledge and relevant skills, often in interactive discourse (Brown, 2004; O’Malley & Valdes-Pierce, 1996) The defining characteristics of teacher-based classroom assessment include (but are not limited to): the use of multiple and 158 complementary instruments, an active role played by the students, the use of meaningful realworld tasks, the exercise of higher-order cognition, an emphasis on the process as well as the product, the integration of various language modalities, in-advance presentation of evaluative criteria to the students (as well as to other stakeholders), and the use of feedback as a way of supporting learning (Brown, 2004; Brown & Hudson, 1998; Fox, 2008; O’Malley & ValdesPierce, 1996) For instance, evaluative criteria in assessment rubrics can highlight what pragmatic aspects are important and need to be learned The criteria can be weighted differently, indicating the different degrees to which each aspect is viewed as crucial in using language appropriately In teacher-based assessment, learners’ competences are typically described or summarized in a descriptive narrative, written in terms of what learners are able to do, as well as what they need to work on further In other words, assessment can provide diagnostic information regarding the learners’ current state of pragmatic competence, while at the same time assisting teachers in determining the subsequent course of pragmatics-focused instruction Diagnostic information is useful especially in formative assessment, which focuses on the learning process Meanwhile, teacher-based assessment also includes summative assessment, in which students’ end product is evaluated for the extent of their achievement Formative assessment aims to determine the level of the students’ learning in progress as part of continued development, and often requires learners’ active involvement in interaction with the teacher and peers (Brown, 2004; Fox, 2008) In contrast, summative assessment is not necessarily designed for future learning but intends to measure or describe how well the learners have accomplished instructional objectives, typically at the conclusion of a course or unit of instruction (Brown, 2004) 159 On the other hand, it is important not to overestimate the credibility and strengths of teacher-based assessment Validity and reliability of classroom-based assessment has been discussed among its proponents (for a detailed discussion, see for example, Brown & Hudson, 1998; Huerta-Macías, 1995; Lynch, 2001; Lynch & Shaw, 2005; McNamara & Roever, 2006) While a certain degree of validity tends to be assumed based on the real-world (or simulated) nature of the tasks being assessed directly (e.g., Huerta-Macías, 1995), some have cautioned that validity should not be taken for granted and that there could be threats to (construct) validity (Brown & Hudson, 1998) Meanwhile, the reliability of teacher-based assessment is often conceptualized differently from traditional standardized testing (Lynch & Shaw, 2005) Rater disagreements may result from differences in raters’ subjective perceptions, and in this case, potential differences in rater reaction can be a natural consequence of pragmatic variation, as argued above In addition, reliability is called into question when a classroom instructor, who may not be professionally trained as a rater, is the sole evaluator of students’ performance working in a time-constrained manner Finally, teacher-based assessment may be seen as relatively impractical, as instruments can be difficult to construct and more time-consuming than those of traditional testing (Fox, 2008) While the implementation of teacher-based assessment can be a daunting task, particularly when L2 pragmatic competence is to be assessed, the challenge does not make the task any less important in the classroom where L2 pragmatics is taught In fact, a major strength of teacher-based assessment is that instruction and assessment are inseparable from each other in a cyclic pedagogical process, serving students’ and teachers’ needs in their immediate instructional contexts This conceptualization of the role of assessment within the learning 160 process can be supported by Vygotsky’s sociocultural framework, which I discuss in the following section 1.3 A theoretical framework for the teacher-based assessment of L2 pragmatics It is important to stress that instruction and teacher-based assessment are one and the same in that they both aim to facilitate learners’ language development Theoretically consistent with this conceptualization of the role of teacher-based assessment is Vygotsky’s sociocultural framework (Fox, 2008; Rea-Dickins, 2008) While interpretations of the Zone of Proximal Development (ZPD) and of its relationship with relevant constructs vary (Lantolf & Thorne, 2006; Ohta, 2005), Vygotsky (1978) originally defined it as the difference between the learner’s developmental level achievable through independent problem solving and the higher level of potential development attainable through problem solving in interaction with others with more advanced cognitive ability, such as a teacher or a more capable peer Closely related to the concept of ZPD is the notion of teacher (or peer) assistance, which is typically mediated by language or cultural artifacts provided to learners based on observation of their competence (e.g., Lantolf, 2000; Lantolf & Thorne, 2006; Ohta, 2005; Wood, Bruner, & Ross, 1976) This notion of mediation is also a defining concept of sociocultural theory as higher forms of our mental activity are regulated by symbolic and physical tools developed by human cultures, including language, gesture, music, art, and numbers (Lantolf, 2000; Lantolf & Poehner, 2008) The learner initially performs tasks under the collaborative guidance of more competent individuals, often mediated through language as part of the process of other-regulation This collaborative inter-mental activity becomes more intra-personal when the learner is capable of self-regulation and achieves independent functioning 161 In teacher-based instruction and the assessment of L2 pragmatics, external assistance may be embedded in the instructional process where teachers direct learners to the key contextual features and lead them to select appropriate language forms Teachers may also point out language features that were not quite appropriate in students’ production to prompt class discussion or self-reflection for improvement Mediation in the form of teacher feedback is dialogically constructed to facilitate understanding and support cognitive development The dialogue may be realized either orally in classroom interaction or through a written channel in a written assignment, for example Teacher feedback may be individualized based on each learner's performance or targeted for a group (or subgroup) of learners in a classroom setting Alternatively, this other-mediation may be provided by more competent peers Through successful other-regulated mediation, learners eventually appropriate and internalize the newly gained knowledge and become self-regulated when they no longer need to rely on outside resources to carry out the task or access that awareness that has now become part of their cognitive repertoire (Lantolf & Thorne, 2006; Vygotsky, 1978) Examples of self- and other-regulation in the area of pragmatic competence can be found in Tateyama and Kasper (2008) and Ohta (2005) In Tateyama and Kasper (2008), the microanalysis of the request episode, in which a classroom guest and a student were engaged, abounds in instances of other-mediation both from the co-performing guest and from the overhearing students While the Japanese native-speaking guest asked the L2 Japanese student to look over her paper written in English, she modified her speech when sensing difficulty More concretely, she shifted between a plain style and a distal masu style perhaps to accommodate the student’s limited proficiency, and she reiterated her request directly to facilitate comprehension After an extended silence, the overhearing peer students supplied the English equivalent of the 162 guest’s utterance and offered the Japanese word with which the student struggled in producing a response The episode demonstrates how these instances of other-mediated assistance guided the student to respond in a more pragmatically appropriate manner in the L2 compared to what the student would achieve without such assistance Ohta (2005) interprets three existing interlanguage pragmatics studies through the sociocultural framework An instance of self-regulation can be observed in the performance of Takahashi’s (2001) Japanese college students in the “explicit” control group, who received teacher-fronted lecture and translation exercises about biclausal requests in English In the posttest discourse completion tasks (DCT), these students showed improvement in their use of the biclausal structures, reaching or approximating the state of self-regulation in terms of producing the target form In Samuda’s (2001) study, self-regulation was observed among Korean and Japanese ESL students in their post-instructional performance The students produced the target modals in both discussion and writing following implicit and explicit scaffolding by the teacher regarding the target modals Ohta identifies this as effective scaffolding addressing the ZPD, as the teacher’s focused instruction was based on the students’ pre-instructional performance, where they relied on non-verbal means and lexical expressions of probability rather than on the target modal forms Similarly, in the initial delivery of oral narratives in Yoshimi (2001), learners of Japanese lacked the use of discourse markers As a result of the teacher’s individually tailored feedback provided in small groups, and of expanded opportunities for the students to compare their own performance with that of expert speakers, the learners’ use of the discourse markers improved While the target discourse markers tended to be phonetically and lexically less salient for learners, the instruction successfully foregrounded these target features and this guided the learners toward self-regulating in the use of these 163 discourse markers in their conversational storytelling The above-mentioned examples in Tateyama and Kasper (2008) and Ohta (2005) are successful instances of pragmatics-focused mediation within the sociocultural framework, although the assessment of the learners’ pragmatic development was measured or observed by researchers rather than by the classroom teachers 1.4 The present study Given the complexities of teacher-based assessment of L2 pragmatic competence as outlined above, exploratory efforts have been made to teach and assess pragmatics in the context of a Japanese-as-a-foreign-language classroom in a U.S university This descriptive case study describes the assessment practices in a regular classroom setting, which were embedded in and inseparable from the instruction It illustrates how learners’ pragmatic development may be born out of written dialogic interaction and how learners’ pragmalinguistic awareness and production might be assessed by teachers at different points in the instruction The instructional and curricular decisions were made by Teacher A, who elected to engage in written dialogue with individual learners about their demonstrated pragmalinguistic competence Written instruments can efficiently elicit learners’ pragmalinguistic awareness, and written records can facilitate both teachers’ assessment and learners’ pragmatic reflective analysis over an instructional period spreading over several weeks Teacher A’s instructional decision appears to be practical in consideration of the demands of many FL instruction settings in the U.S In this study, learners’ language use during the first seven weeks was assessed formatively with the intention to explore how the teachers’ supportive assistance may have mediated the learners’ pragmatic development through written collaboration over this period In 164 which guided the learners’ self-reflection on their own work Through this process of otherregulation, learners were sometimes able to produce more target-like pragmalinguistic structures With the supportive formative assessment provided over the weeks of instruction, learners appear to have become able to vary their pragmatic use of language according to the social context to different degrees A close look at learners’ pragmatic performance in Week and Week and their reflections in Week reveals a range of pragmatic development Learner #1 demonstrated the use of the appropriate request perspective and general use of the plain style in appropriate contexts At the same time, close examination reveals that his pragmalinguistic awareness of the plain or distal style choice, which was probably enhanced by the preceding interactions with the teacher, was not necessarily accompanied by appropriate pragmalinguistic production While Learner #1 was able to produce the preferred expression, wasure chatta ‘I happened to forget (plain form)’ in his teacher-supported reflection in Week 7, which is generally more appropriate for use with a close friend as in this scenario, he failed to use the plain form in the same DCT a week later In Week 8, although he generally used the more context-appropriate plain form, in this particular case, he reverted back to a more formal register wasurete shimatte ‘I happened to forget (distal form)’, the style he used across both formal and informal DCT situations in Week This may mean that although he largely took advantage of the mediated interaction provided in the reflective assignment in Week 7, his learning may not have been fully internalized or self-regulated In order for this pragmalinguistic knowledge to be completely self-regulated, he may well have needed further assisted productive practice or more fine-tuned pragmalinguistic awareness supported by external artifacts or resources On the other hand, the same learner demonstrated the ability to produce at least two appropriate forms of request in response to the given scenario in Weeks 7‒8 While his reflection 181 in Week shows his pragmalinguistic control and understanding of the permissive form, tsukawasete kurenai ‘can you let me use (plain form),’ what he actually used in Week was another appropriate expression, kashite moraenai ‘can I borrow (plain form).’ As with the previous case of plain and distal speech styles, he may not have completely internalized the pragmalinguistic form, tsukawasete kurenai ‘can you let me use (plain form)’ and he may have failed to demonstrate its self-regulated use in Week He may have needed further assistance to be able to produce the form, which was still going through the process of other-regulation a week earlier Alternatively, the learner may have developed the understanding of the request perspective through the teacher-student interaction during Week 7, which directed the learner’s attention to the request perspective (4, 5) Another expression using another perspective, kashite moraenai ‘can I borrow (plain form)’ was also introduced and practiced between Weeks and 7, and the learner appears to have self-regulated its use by Week In this interpretation it is still unclear whether he has also internalized the form tsukawasete kurenai ‘can you let me use (plain form).’ However, the learner seems to have self-regulated the awareness of the request perspective and the structure of the permissive form with the self-regulated production of at least one of the forms, tsukawasete kurenai ‘can you let me use (plain form).’ While the exact extent of Learner #1’s cognitive development remains unclear in this study, he probably accumulated noticeable effects of mediational benefits over the weeks of instruction and demonstrated some of this in his written work in Week The teachers’ narrative and quantified assessment of his work made some of these challenges and this progress identifiable to both the teachers and the learner Although Learner #7’s pragmatic learning may appear to be less recognizable in terms of the ratings she received, her enhanced pragmatic awareness demonstrated in her Week reflection was highly consistent with her demonstrated 182 pragmatic production a week later Compared to Learner #1, she may have self-regulated much of what she was made aware of through the mediational dialogue and was able to demonstrate this awareness, which had become part of her cognitive repertoire It should be added that the focal learners were no exceptional cases Rather, other learners also demonstrated enhanced pragmalinguistic competence as well as areas that needed further improvement as assessed by the teachers through the exploratory assessment instruments used in this study 4.1 Limitations of this study and suggestions for further research Because there have not been many precedents in assessing pragmatic competence in the classroom setting, the exploratory attempts reported in this study were often made as the instruction and assessment progressed The evaluative criteria in the rubrics and some of the assessment instruments went through multiple revisions and refinements This process required a recursive cycle of pilot-testing, examination of actual learner responses, and negotiation of the criteria among the rating teachers and the researcher While this labor-intensive process of refining assessment instruments may be relevant to adjustments required in other instructional settings, the quantified data in this study have no statistical significance and the findings are not easily applicable to other settings In future research, statistical analysis of the reliability of summative assessments made by classroom teachers could be conducted with more raters and with larger samples of learner language The assessment instruments could also be refined further Evaluative criteria may need to be assigned more points or descriptions in order to discriminate learner performance more sensitively Depending on the emphasis given to different aspects of pragmatics during instruction, evaluative criteria could also be weighed to reflect the varied instructional foci 183 Another major limitation of this study is that the data examined were limited to written interactional data as projective measures of the learners' oral skills In the Vygotskian framework, interaction, through which higher-level cognitive development can occur, does include interaction through a written channel Analysis of written language also facilitates reflective assessment for both teachers and learners and is realistic in the context of the classroom However, written data may be reflective of learners’ (meta)pragmatic awareness rather than of their real-time use of language Learners’ oral interaction with their peers and teachers could be video-recorded in future research so that verbal and non-verbal language and the process of cognitive learning may be assessed more directly and holistically Ideally, learners’ interactions would be recorded through multiple instruments and observed in authentic interactions as well as in classroom discourse, although such detailed analyses will likely go beyond the scope of classroom-based assessment Another limitation of the classroom-based assessment reported in this study is that the assessment relied heavily on assessor intuition As a result, a full range of pragmatic variation was not incorporated into the assessment in the way that data-driven assessment would have permitted Learners’ pragmatic awareness and use of language was largely compared to what the teachers imagined most Japanese speakers would say or what they should say rather than to what college-age Japanese students would actually say As a result, a descriptive range of pragmatic variation may not have been used as a baseline in the assessment For example, in the L2 Japanese classroom, a pedagogical convention is to associate the plain style with informality and the distal desu/masu style with formality (Ishida, 2005) Learners are typically instructed to use the distal style for older or higher-status interlocutors (meue) and the plain style for younger or lower-status interactants (meshita) (Ishida, 2005) However, in authentic contexts, pragmatically 184 competent speakers of Japanese have been found to dynamically style-shift between these forms across different modes of communication to index a range of pragmatic meanings (Cook, 2001, 2008; Ishida, 2005; Maynard, 1991; Tateyama & Kasper, 2008) Although in this study learners were instructed to take a wider range of contextual factors (e.g., social and psychological distance, gender, and level of imposition) into account, rather than just age and relative social status, the idealized distinction between these forms was applied in the assessment Such conventional instruction and assessment may reflect the assessors’ perception of socially desirable behavior, but at the same time it can mislead learners into believing that there is a static one-to-one correspondence between these forms and seemingly dichotomous levels of formality In future instruction of Japanese pragmatics, teachers and learners could observe a range of linguistic strategies that pragmatically competent speakers use in order to study the subtle pragmatic nuances they intend to convey in Japanese (see the pedagogical suggestions for teaching the distal and plain styles in Ishida, 2005; also see Yoshimi, 2001) The insights into pragmatic norms and variation obtained in such observations as well as findings informed by current research (e.g., Walters, 2009) can provide a point of reference in assessment While teachers’ intuition as to what is likely to sound more or less appropriate in a given context can be an important yardstick in assessment, teachers could also draw on more authentic (e.g., naturally occurring or role-played) interactions in the comparable contexts and use the demonstrated range of pragmatic norms as part of the baseline in assessment (see Ishihara & Cohen, 2010 for examples) In order to implement this type of instruction and assessment grounded in research and enhanced instructional practice, teachers may need to be specifically prepared for pragmatics-focused instruction and assessment as part of their professional development 185 Conclusion Using Vygotsky’s sociocultural theory, this paper has attempted to show the way in which learners’ pragmalinguistic awareness and production were mediated through dialogue with the teachers The collaborative dialogues utilized in the formative assessment successfully supported the process of self-regulation for some of the pragmalinguistic structures, which the learners were able to demonstrate in the summative assessment a week later However, the same summative assessment showed that the learning of some pragmalinguistic forms failed to fulfill complete internalization and would have required further other-regulation It is promising that the learners’ degree of pragmalinguistic development became tangible to teachers through their use of the formative and summative assessment instruments proposed here Because learners’ pragmatic awareness can be tapped and their pragmatic development may be triggered through instructional interactions, classroom-based assessment of L2 pragmatics used for formative purposes may be implemented effectively in the collaborative learning environment The summative assessment appeared to have left several issues to be considered especially in terms of rater reliability At the same time, these issues are related to the challenge of how pragmatic language use, whose benchmark is varied and dynamic in nature, can be quantified and assessed not by the researcher in the laboratory setting but by teachers in the classroom with all its instructional and institutional constraints References ACTFL (1999) ACTFL proficiency guidelines: Speaking Retrieved May 13, 2013, from http://www.sil.org/lingualinks/LANGUAGELEARNING/OtherResources/ACTFLProfici encyGuidelines/ACTFLGuidelinesSpeaking.htm 186 Ahn, R C (2005) Five measures of interlanguage pragmatics in KFL (Korean as a foreign language) learners Unpublished doctoral dissertation, University of Hawai‘i Beebe, L M., & Waring, H Z (2004) The linguistic encoding of pragmatic tone: Adverbials as words that work In D Boxer & A D Cohen (eds.), Studying Speaking to Inform Second Language Learning (pp 228‒249) Clevedon: Multilingual Matters Brown, J D (2001) Pragmatics tests: Different purposes, different tests In K R Rose & G Kasper (eds.), Pragmatics in Language Teaching (pp 301‒325) Cambridge: Cambridge University Press Brown, H D (2004) Language Assessment: Principles and classroom practices New York: Longman Brown, H D., & Hudson, T (1998) The alternatives in language assessment TESOL Quarterly, 32(4): 653‒675 Canale, M., & Swain, M (1980) Theoretical bases of communicative approaches to second language teaching and testing Applied Linguistics, 1(1): 1‒47 Cohen, A D., & Shively, R., L (2003) Measuring speech acts with multiple rejoinder DCTs Language Testing Update, 32: 39‒42 Cook, H M (2001) Why can't learners of JFL distinguish polite form impolite speech styles? In K R Rose & G Kasper (eds.), Pragmatics in Language Teaching (pp 80‒102) Cambridge: Cambridge University Press Cook, H M (2008) Socializing identities through speech style: Learners of Japanese as a foreign language Clevedon: Multilingual Matters Cumming, A (2009) Language assessment in education: Tests, curricula, and teaching Annual Review of Applied Linguistics, 29: 90‒100 187 Davison, C., & Leung, C (2009) Current issues in English language teacher-based assessment TESOL Quarterly, 43(3): 393‒415 Enochs, K., & Yoshitake-Strain, S (1999) Evaluating six measures of EFL learners' pragmatic competence JALT Journal, 21(1): 29‒50 Fox, J (2008) Alternative assessment In E Shohamy & N H Hornberger (eds.), Encyclopedia of Language and Education: Language testing and assessment (2nd edn, Vol 7, pp 97‒108) New York: Springer Hartford, B S., & Bardovi-Harlig, K (1992) Experimental and observational data in the study of interlanguage pragmatics Pragmatics and Language Learning, 3: 33‒52 Hudson, T (2001) Indicators for pragmatic instruction: Some quantitative tools In K R Rose & G Kasper (eds.), Pragmatics in Language Teaching (pp 283‒300) Cambridge: Cambridge University Press Hudson, T., Detmer, E., & Brown, J D (1992) A Framework for Testing Cross-cultural Pragmatics Honolulu, HI: University of Hawai‘i at Mānoa, Second Language Teaching & Curriculum Center Hudson, T., Detmer, E., & Brown, J D (1995) Developing Prototypic Measures of Crosscultural Pragmatics Honolulu, HI: University of Hawai‘i at Mānoa, Second Language Teaching & Curriculum Center Huerta-Macías, A (1995) Alternative assessment: Responses to commonly asked questions TESOL Journal, 5(1): 8-11 Iino, M (1996) "Excellent foreigner!" Gaijinization of Japanese language and culture in contact situations: An ethnographic study of dinner table conversations between Japanese host families and American students Unpublished doctoral dissertation, 188 University of Pennsylvania Ishida, K (2005) Why shift forms when addressing the same person? Raising awareness about the pragmatic use of the Japanese plain and desu/masu forms In D Tatsuki (ed.), Pragmatics in Language Learning, Theory, and Practice (pp 161‒165) Tokyo: The Japan Association for Language Teaching, Pragmatics Special Interest Group Ishihara, N (2007) Web-based curriculum for pragmatics instruction in Japanese as a Foreign Language: An explicit awareness-raising approach Language Awareness, 16: 21‒40 Ishihara, N (2009) Teacher-based assessment for foreign language pragmatics TESOL Quarterly, 43(3): 445‒470 Ishihara, N., & Cohen, A D (2010) Teaching and Learning Pragmatics: Where language and culture meet Harlow: Pearson Education Ishihara, N., & Maeda, M (2010) Advanced Japanese: Communication in context [Kotobato bunkano kousaten: Bunkade yomitoku nihongo] London: Routledge Ishihara, N., & Tarone, E (2009) Subjectivity and pragmatic choice in L2 Japanese: Emulating and resisting pragmatic norms In N Taguchi (ed.), Pragmatic Competence (pp 101‒128) Berlin: Mouton de Gruyter Itomitsu, M (2009) Developing a test of pragmatics of Japanese as a foreign language Unpublished doctoral dissertation, Ohio State University, Columbus, OH Johnston, B., Kasper, G., & Ross, S (1998) Effect of rejoinders in production questionnaires Applied Linguistics, 19(2): 157‒182 Kasper, G., & Dahl, M (1991) Research methods in interlanguage pragmatics Studies in Second Language Acquisition, 13: 215‒247 Kasper, G., & Rose, K R (2002) Pragmatic Development in a Second Language Malden, MA: 189 Blackwell Publishing Kinginger, C (2002) Defining the zone of proximal development in US foreign language education Applied Linguistics, 23(2): 240‒261 Lantolf, J P (2000) Sociocultural Theory and Second Language Learning Oxford: Oxford University Press Lantolf, J P., & Poehner, M E (2008) Dynamic assessment In E Shohamy & N H Hornberger (eds.), Encyclopedia of Language and Education: Language testing and assessment (2nd edn, Vol 7, pp 273‒284) New York: Springer Lantolf, J P., & Thorne, S L (2006) Sociocultural Theory and the Genesis of Second Language Development Oxford: Oxford University Press Leung, C (2007) Dynamic assessment: Assessment for and as teaching? Language Assessment Quarterly, 4(3): 257‒278 Liu, J (2006) Measuring Interlanguage Pragmatic Knowledge of EFL Learners Frankfurt: Peter Lang Publishing Lynch, B K (2001) Rethinking assessment from a critical perspective Language Testing, 18(4): 351‒372 Lynch, B., & Shaw, P (2005) Portfolios, power, and ethics TESOL Quarterly, 39(2): 263‒297 Maynard, S K (1991) Pragmatics of discourse modality: A case of da and desu/masu forms in Japanese Journal of Pragmatics, 15(6): 551‒582 McNamara, T F., & Roever, C (2006) Language Testing: The social dimension Malden, MA: Blackwell 190 Merriam, S B (1998) Qualitative Research and Case Study Applications in Education: Revised and expanded from Case Study Research in Education San Francisco, CA: Jossey-Bass Publishers Ochs, E (1996) Linguistic resources for socializing humanity In J Gumperz & S Levinson (eds.), Rethinking Linguistic Relativity (pp 407‒437) Cambridge: Cambridge University Press Ohta, A S (2000) Re-thinking interaction in SLA: Developmentally appropriate assistance in the zone of proximal development and the acquisition of L2 grammar In J P Lantolf (ed.), Sociocultural Theory and Second Language Learning (pp 51‒78) Oxford: Oxford University Press Ohta, A S (2005) Interlanguage pragmatics in the zone of proximal development System, 33: 503‒517 O'Malley, M., & Valdez Pierce, L (1996) Authentic Assessment for English Language Learners White Plains, NY: Addison-Wesley Publishing Rea-Dickins, P (2008) Classroom-based language assessment In E Shohamy & N H Hornberger (eds.), Encyclopedia of Language and Education: Language testing and assessment (2nd edn, Vol 7, pp 257‒271) New York: Springer Science+Business Media LLC Rintell, E M., & Mitchell, C J (1989) Studying requests and apologies: An inquiry into method In S Blum-Kulka, J House, & G Kasper (eds.), Cross-cultural Pragmatics: Requests and apologies (pp 248‒272) Norwood, NJ: Ablex Publishing Corporation Roever, C (2005) Testing ESL Pragmatics: Development and validation of a web-based assessment battery Frankfurt: Peter Lang 191 Roever, C (2008) Rater, item, and candidate effects in discourse completion tests: A FACETS approach In E S Alcón & A Martínez-Flor (eds.), Investigating Pragmatics in Foreign Language Learning, Teaching, and Testing (pp 249‒266) Bristol: Multilingual Matters Rose, K R (1994) On the validity of discourse completion tests in non-Western contexts Applied Linguistics, 15(1): 1‒14 Samuda, V (2001) Guiding relationships between form and meaning during task performance: The role of the teacher In M Bygate, P Skehan, & M Swain (eds.), Researching Pedagogic Tasks: Second language learning, teaching, and testing (pp 119‒140) New York: Longman Schneider, K P., & Barron, A (eds.) (2008) Variational Pragmatics Amsterdam: John Benjamins Siegal, M (1996) The role of learner subjectivity in second language sociolinguistic competency: Western women learning Japanese Applied Linguistics, 17(3): 356‒382 Taguchi, N (2011) Rater variation in the assessment of speech acts Pragmatics, 21(3): 453‒471 Taguchi, N (2012) Context, Individual Differences and Pragmatic Competence Bristol: Multilingual Matters Takahashi, S (2001) The role of input enhancement in developing pragmatic competence In K R Rose & G Kasper (eds.), Pragmatics in Language Teaching (pp 171‒199) Cambridge: Cambridge University Press Tateyama, Y., & Kasper, G (2008) Talking with a classroom guest: Opportunities for learning Japanese pragmatics In E.S Alcón & A Martínez-Flor (eds.), Investigating Pragmatics in Foreign Language Learning, Teaching, and Testing (pp 45‒71) Bristol: Multilingual Matters 192 Vygotsky, L S (1978) Mind in Society: The development of higher psychological processes Cambridge, MA: Harvard University Press Walters, F S (2009) A conversation analysis-informed test of L2 aural pragmatic comprehension TESOL Quarterly, 43(1): 29‒54 Wood, D., Bruner, J., & Ross, G (1976) The role of tutoring in problem solving Journal of Child Psychology and Psychiatry, 17(2): 89‒100 Yamashita, S O (1996) Six Measures of JSL Pragmatics Honolulu, HI: University of Hawai‘i at Mānoa, Second Language Teaching & Curriculum Center Yoshimi, D R (2001) Explicit instruction and JFL learners' use of interactional discourse markers In K R Rose & G Kasper (eds.), Pragmatics in Language Teaching (pp 223‒244) Cambridge: Cambridge University Press 193 Appendix: Multiple-rejoinder DCTs Read carefully from beginning to end each of the following dialogue situations Then take several minutes to think out in your mind just what you would say in such a situation while interacting with fluent speakers of Japanese in Japan You may want to think about speakers of Japanese you know personally and imagine how you would interact with them Once you have a clear idea of what you would say, complete the dialogue [Names are intended to be unisex so that the students can choose the gender in the role-play practice.]  Akira is going to enter a speech contest next month S/he has been very motivated to a good job and has been working hard on the draft S/he hears that one of your friendly junior teachers, Professor Murata, has served as a judge before and decides that it would be good to get her advice on both the draft and the delivery of the speech After class, s/he approaches Professor Murata and asks if she could help Akira: Professor Murata : ええ、 どう しましたか。 Akira: Professor Murata : いいですよ。 じゃあ来週のオフ ィ スアワーにどう ですか。 Akira:  Shinobu is in a classroom in college, waiting for class to begin any time soon Then, s/he notices that s/he forgot to call his/her boss at his/her part-time job to let her know his/her 194 schedule this week Shinobu turns to a close friend, Satomi, who is sitting right next to him/her, to borrow her cell phone for a quick call Shinobu: Satomi: ああ、 いいけど、 いそいでる？ Shinobu: Satomi: じゃあ、 これ。 終わっ たら電源切っ と いてね。 Shinobu: 195 ... of instruction (Brown, 2004) 159 On the other hand, it is important not to overestimate the credibility and strengths of teacher- based assessment Validity and reliability of classroom -based assessment. .. section 1.3 A theoretical framework for the teacher- based assessment of L2 pragmatics It is important to stress that instruction and teacher- based assessment are one and the same in that they... conceptualization of the role of teacher- based assessment is Vygotsky’s sociocultural framework (Fox, 2008; Rea-Dickins, 2008) While interpretations of the Zone of Proximal Development (ZPD) and of its