Do They Make a Difference? The Impact of English Language Programs on Second Language Students in Canadian Universities

Do They Make a Difference? The Impact of English Language Programs on Second Language Students in Canadian Universities JANNA FOX Carleton University Ottawa, Canada LIYING CHENG Queen’s University Kingston, Canada BRUNO D ZUMBO University of British Columbia Vancouver, Canada Few studies have investigated the impact of English language programs on second language (L2) students studying in Canadian universities (Cheng & Fox, 2008; Fox, 2005, 2009) This article reports on questionnaire responses of 641 L2 students studying in 36 English language programs in 26 Canadian universities The researchers identified programs by their activity emphasis as either English as a second language (ESL) or English for academic purposes (EAP) Activity emphasizing speaking, social interaction, and general language development was viewed as ESL, whereas activity that emphasized academic reading, writing, and language development was considered EAP The researchers used structural equation modeling procedures to examine the network of relationships between language program emphasis and participants’ background characteristics in influencing academic and social engagement A model of moderated mediation (Wu & Zumbo, 2008) was confirmed; that is, language program activities were found to account for variation in strategies which mediated academic and social engagement However, the impact was moderated (lessened or strengthened) by three personal background factors: anxiety, stress, and motivation This study refines our understanding of the positive impact of ESL and EAP programs on L2 university students’ academic and social engagement doi: 10.1002/tesq.103 TESOL QUARTERLY Vol 48, No 1, March 2014 © 2013 TESOL International Association 57 I n response to the increasing numbers of second language (L2) learners studying in Canadian universities, many new language programs have been developed These programs tend to share the common goal of developing L2 learners’ English language in order to support their successful transition to and engagement with academic work in university (Cheng & Fox, 2008; Fox, Cheng, Berman, Song, & Myles, 2006), but they differ considerably in emphasis, methodology, and approach Surveys of such programs across Canada have demonstrated little consistency (Berman, 2002) and less evidence of their specific contribution in supporting L2 students’ engagement with the disciplinary work of the university Indeed, these surveys (Berman, 2002; Cheng & Fox, 2008) reveal a patchwork of programs, which may be roughly classified into two types: English as a second language (ESL; general English) or English for academic purposes (EAP) We use ESL rather than English for speakers of other languages (ESOL) or English as an additional language (EAL), because at the time of this study most Canadian universities continued to label their language support courses as ESL and/or EAP Previous research suggests that ESL programs tend to focus on overall language development and conversational English They tend to utilize a range of teacher-made thematic or textbook units in order to boost students’ acquisition of idioms, grammar, vocabulary, metaphor, and so forth, across a range of different communicative situations (e.g., booking travel in Canada, managing a meeting) and texts (e.g., magazine/newspaper articles, radio or television news broadcasts, film) The goals of ESL programs are typically related to increasing communicative capability (Savignon, 2002) and tend to emphasize speaking activities and social interaction (e.g., role-plays, conversation gaps, group work) while fostering a sense of belonging, community, and connection with the new (Canadian) context and culture EAP programs, on the other hand, are explicitly concerned with the skills and strategies in English that are “required for study purposes in formal education systems” (English-Teaching Information Centre, cited in Jordan, 2004, p 1) They have typically been characterized by activities that simulate academic work; for example, reading academic texts and taking notes; writing research reports or essays; making formal seminar presentations; and acquiring strategies to increase comprehension by scanning, skimming, and evaluating academic texts (Cheng, Myles, & Curtis, 2004) EAP programs aim to develop a “general academic English register, incorporating a formal, academic style with proficiency in the language use” (Jordan, 2004, p 5) 58 TESOL QUARTERLY The impact of such ESL and EAP language programs has remained underexamined, however At present, in the face of economic pressures, cutbacks, funding shortages, and accountability agendas, many English language programs are being confronted with the need to provide evidence of their effectiveness (Fox, 2009; Swales, Barks, Ostermann, & Simpson, 2001) However, generally such programs have no coherent (or convincing) research to fall back on for support Further, over the past decade, the pedagogical value of EAP approaches has been questioned by researchers (e.g., Freedman, 1999; Hansen, 2000) who argue that language is embedded in discipline-specific academic and professional contexts These researchers maintain that language courses which focus on general academic knowledge, skills, or strategies may be of little value to students, who must meet the narrower demands of specific disciplinary academic cultures Further, some have argued that English language teaching has not been sufficiently critical of covert political and economic influences that have motivated trends in the field (Pennycook, 1996) “They point out, for example, that what is considered plagiarism by EAP teachers and the dominant academic culture they represent devalues L2 students’ academic and professional experiences in their native countries” (Cheng & Fox, 2008, p 312) The interaction of students’ personal background characteristics with program emphasis and activity has also been underexamined (Berman, 2002; Cheng & Fox, 2008; Fox, 2004; Fox et al., 2006) Thus there remain many unanswered questions For example, both ESL and EAP activities have the same impact on academic engagement? Do ESL and EAP programs increase L2 students’ satisfaction with the quality of life of their new academic study cultures? What is the relationship between program type, individual students’ background characteristics, and academic and social engagement? This study begins to address these questions by investigating the impact of English language support programs on L2 students studying in Canadian universities Specifically, the study examined (1) the relationship between language program activity and student background characteristics and (2) the impact of that relationship on academic engagement and satisfaction with life in Canada, operationalized in the study as social engagement The results of the study highlight differences between variables that mediated the impact of language program activity on student engagement (i.e., directly linked program emphasis and outcomes to impact) and those that moderated engagement (i.e., lessened or strengthened program impact) Before considering the study itself, in the following section we provide an overview of literature that has examined factors that affect L2 students’ academic and social engagement in other research contexts IMPACT OF ENGLISH LANGUAGE PROGRAMS IN CANADIAN UNIVERSITIES 59 FACTORS THAT IMPACT THE ACADEMIC AND SOCIAL ENGAGEMENT OF L2 STUDENTS Many factors that may potentially impact L2 student learning have been examined in the research literature Some studies have focused on language proficiency as a predictor of academic achievement, often indicated by grade point averages (GPAs) at the end of a course or at program completion, or by the perceptions of faculty and students However, as Christopher (1993) points out, GPAs not take into account students who may have dropped out, nor they allow for disciplinary differences Other research suggests that language acquisition variables such as time in an English-dominant country, type of high school attended, and previous university experience are good predictors of future academic achievement—and better than sociodemographic factors (i.e., gender, parents’ education, or ethnicity) or standardized test scores (see, e.g., Fox, 2009; Shih & Brown, 2000) Some research (e.g., Johnson, 1988) has found that standardized proficiency tests (e.g., Test of English as a Foreign Language, TOEFL) correlate with outcomes such as academic credits earned and program completion; however, such studies have been questioned (Cotton & Conrow, 1998; Fox, 2004; Kerstjens & Nery, 2000), because so many other factors potentially affect academic outcomes (e.g., motivation, educational background, field of study, family pressures, support) Macrae (1997) argued that test scores need to be systematically examined in relation to such factors as age, motivation, language, and educational and cultural background Further, Leki (1995) found evidence that new cultural contexts—both academic and social— impact student performance (for a review of these studies and factors which impact L2 students in English-medium universities, see Cheng & Fox, 2008; Fox et al., 2006) Al-Sharideh and Goe (1998) identify social connections and/or friendship as a key factor in the academic engagement process, finding that L2 students needed to find a balance between their friendships with students from their own L2 cultural groups with local student friendships Their research highlights the importance of a social network in academic retention and success Their findings are supported by Scanlon, Rowling, and Weber’s research (2007), which reports on transition to university as a “loss experience” (p 223), arguing that new university students often undergo a type of identity discontinuity They found that new students in transition to university, regardless of language or cultural background, typically drew on knowledge of past learning contexts in their attempts to negotiate new identities as university students, but that this was generally unhelpful 60 TESOL QUARTERLY Their findings suggest the importance of social interaction in making an effective transition to university: “Identity results from situated interactions in which students pick up cues regarding the horizons of possibility for identity formation in university transition” (p 223) They argue that “it is the nexus of situated [social] interactions with lecturers and other students that is the context and process of identity formation” (p 223) These studies suggest that any examination of students’ academic engagement process must be related to considerations of their social engagement, and whether or not effective or satisfying social connections are being made A number of large-scale surveys have thus investigated student perceptions of the relative importance of language, academic, and social skills (Leki, 2001) For example, Berman and Cheng (2001) reported that the most difficult academic language skills for L2 students in a Canadian university were those that required either oral communication (e.g., group work, presentations, class discussions) or writing (e.g., research essays, reports) Further, a number of studies have examined individual L2 students’ written and spoken accounts of their transition to university-level study (e.g., Ivanic, 1998, 2006; Johns, 1997; Leki, 1995) and documented how student identities were constructed over time These studies have provided valuable insights regarding the role of language program support In addition, in recent years there has been an increasing number of longitudinal and large-scale studies of EAP program support Fox (2004, 2005) provided evidence of the role of EAP in L2 students’ engagement with university in longitudinal tracking studies that compared two cohorts of L2 students (one that had EAP support and one that did not), with their native-English-speaking counterparts In both studies, L2 students with language support had a much greater likelihood of academic success Banerjee and Wall (2006) reported on a survey of U.K institutions’ presessional EAP courses and the role of tests in acceptance and placement They discussed challenges faced by EAP programs in determining what constitutes successful completion and concomitant readiness for academic study, and provided a checklist of criteria to consider in assessing the academic readiness of presessional EAP students Green (2007) examined the impact of the International English Language Testing System (IELTS) on the provision of EAP courses and the relationship between proficiency, test preparation, and the development of academic literacy, finding evidence that many presessional programs have felt pressure to measure gains in proficiency using IELTS as the outcome He also reported on a disconnect between the content of the test and the objectives of most presessional EAP courses, which rendered such expectations for gains in proficiency unrealistic Moore and Morton (2005) also noted that in many Australian institutions presessional IMPACT OF ENGLISH LANGUAGE PROGRAMS IN CANADIAN UNIVERSITIES 61 EAP courses have a dual purpose: to prepare students both for university study and for the IELTS The studies cited here suggest that a range of academic, social, and program factors may impact an L2 student’s engagement with university study They also suggest a growing awareness of the complexity of language learning and a burgeoning interest in differing approaches to language program support HYPOTHETICAL MODEL OF IMPACT The present study addressed the need for evidence of the impact of language support on L2 students’ academic and social engagement in Canadian universities It was informed by the results of a qualitative study (Cheng & Fox, 2008; Fox et al., 2006) that reported on semistructured interviews with 56 students attending three Canadian universities, who were asked about their experiences in ESL, EAP, and disciplinary courses In accounting for their transition to the academic demands of university study in Canada, the participants demonstrated, as Morita (2000) points out, that transition is “complex, locally situated [and] involves dynamic negotiations of expertise and identity” (p 304) However dynamic and situated this process may be, Cheng and Fox (2008) identify recurring patterns of response across the theoretical samples (Charmaz, 2006) drawn from the three research sites These were reduced to seven major themes (subcategories) and three categories (Figure 1): Academic and social engagement (e.g., using/developing learning strategies, accessing social/academic resources, adapting to individual field of study) Student characteristics (e.g., anxiety, stress, motivation, language and academic background) Language program characteristics (e.g., ESL, EAP) These results suggest that language program support activities (ESL or EAP) may be linked to academic engagement and satisfaction with life in Canada, and that the academic acculturation process is mediated by course outcomes (e.g., increased English language use, strategy use, goal setting) At the same time, the results suggest that the impact of language support is moderated (i.e., strengthened or lessened) as a result of a student’s personal characteristics (e.g., motivation, stress due to external pressures, individual [internal] feelings of anxiety) We hypothesized that the relationship between language support program characteristics and academic or social engagement might best 62 TESOL QUARTERLY Themes Categories Using or developing learning strategies Accessing social/academic resources Academic and Social Engagement Adapting to individual fields of study Academic motivation Language and academic background Student Characteristics Personal affect (anxiety or emotional state) EAP and ESL program support Language Program Characteristics FIGURE Themes (subcategories) and categories in L2 students’ accounts of their academic acculturation processes (N = 56) (Cheng & Fox, 2008) Note The information in Figure is reprinted with permission from The Canadian Modern Language Review/La Revue Canadienne des Langues Vivants, 65, 275–305 [doi:10.3138/cmlr.65 2.307] © 2008 University of Toronto Press be described as one of moderated mediation Models of moderated mediation are well documented in the research literature (cf Muller, Judd, & Yzerbyt, 2005; Wu & Zumbo, 2008) and are used in a variety of disciplines (e.g., physics, medicine, business, psychology) to explain complex interactions involving multiple variables: The moderated mediation model hypothesizes that the mediation effect, which is responsible for producing the effect of the independent variable [i.e., language program characteristics] on the dependent variables [i.e., academic and social engagement] depends on the value or level of the moderator [i.e., student characteristics] (Wu & Zumbo, 2008, p 384) Whereas the mediating variables might explain why and how the relationship occurred (in this case, language support brought about changes in students’ language use, strategy use, and goal-directed practices, and those changes in turn affected students’ academic and social engagement), the moderating variables might explain when or for whom (Wu & Zumbo, 2008), based on students’ levels of anxiety, stress, and motivation Figure provides an illustration of the hypothesized moderated mediation model accounting for the impact of ESL or EAP program characteristics on academic and social engagement In the diagram, the direct effect of English language program on academic and social engagement is illustrated by path c The role of course outcomes in mediating academic and social engagement is in bold The mediation effect (illustrated by paths a and b) is moderated by student characteristics, which either lessen or strengthen program impact Moderating effects on a and b are demonstrated by the arrows with shading IMPACT OF ENGLISH LANGUAGE PROGRAMS IN CANADIAN UNIVERSITIES 63 FIGURE Hypothesized theoretical relationship of moderated mediation effects of language program characteristics on academic and social engagement In examining the hypothesized theoretical relationships in Figure 2, we specifically sought to answer the following three questions: Is there evidence that ESL and EAP programs have a direct effect on L2 students’ academic and social engagement? Do course outcomes (defined in this study as goal-setting and other goal-directed practices, English language use, and strategy use) mediate academic and social engagement? If so, does mediation differ in relation to program emphasis? Do student characteristics (i.e., levels of anxiety; stress caused by pressure from parents, finances, etc.; and motivation) moderate course outcomes? Does moderation differ in relation to program characteristics (ESL or EAP)? If so, how does it differ? METHOD Participants A questionnaire (see the Appendix) was distributed to English-medium language support programs in universities across Canada All university language programs which were members of official organizations (e.g., TESL Canada, TESL Ontario, Alberta-TESL) and/or were mentioned on official university websites were included in the survey 64 TESOL QUARTERLY Only English-medium universities in Canada were included in this study It is important to point out that many Canadian universities have a number of separate and independent programs to support L2 students, which are housed within different faculties, departments, or units Further, in larger Canadian universities, there are many ad hoc language support programs which have arisen in response to local needs within units, departments, or faculties These ad hoc programs were not included in the present survey Fully completed questionnaires were received from 641 L2 students studying in 36 language support programs (ESL and/or EAP) in 26 Canadian universities Of the 641 participants, 80% (n = 513) were international students, with the remaining 20% (n = 128) permanent residents of Canada or new immigrants They spoke 44 different first languages, with 80% speaking only one additional language (English) The most often reported first languages in order of frequency were Chinese, Korean, Japanese, Arabic, and French Most respondents were 20–22 years old at the time of the study (n = 269 or 42%), although participants indicated they were 17, and 10 indicated they were 40 or older The mean age of the respondents was 22.3 years (SD = 4.9); 359 (56%) indicated that their study in Canada was supported by their family; 51 (7%) reported they were fully supported by a scholarship; another 52 (8%) students indicated they had received loans to support their study; and 43 (6%) reported that they were working in order to support their study in Canada The remaining 136 (21%) indicated that they received some combination of support from, for example, family and scholarship, family and work, work and loan, and so forth Eighty-six percent (n = 550) indicated that their parents had some form of tertiary-level education (college or university) The largest number of respondents, 325 (50.7%), were studying in EAP programs at the time of the study; 255 (39.8%) were studying English in pre-university ESL intensive programs, and 27 (4.2%) had recently completed EAP and were newly enrolled in undergraduate study; 34 (5.3%) did not indicate the type of language support they had or were receiving Instruments A questionnaire (see the Appendix) was designed to investigate the relationships illustrated in Figure In the first section of the questionnaire, Items 1–7 elicited demographic information (e.g., age, gender, first language [L1], intended program of study, grades, parents’ education); in Section 2, Items 8–19 asked students to identify the IMPACT OF ENGLISH LANGUAGE PROGRAMS IN CANADIAN UNIVERSITIES 65 relative importance of factors influencing their choice of university (e.g., career goals, friends, parents) and to provide additional background information (e.g., about financial support, schooling, work experience) The third section, consisting of 36 Likert-scale items, elicited responses regarding motivation, stress, anxiety, strategy use, language use (L1 vs L2), goals, and satisfaction with academic and social life in Canada In the fourth section participants responded to 10 Likert-scale items regarding activity in their L2 English support course and to open-ended items, which elicited additional information about program, plans, and the usefulness of their language support These items are the focus of a separate study (see Cheng & Fox, 2008) The Likert-scale items in Sections and were of particular importance in testing the validity of the hypothesized model (Figure 2) They were, in effect, operational definitions of each of the variables in the model For example, language use was operationalized by items such as 43 (I spend most of my time speaking English) and 45 (I spend most of my time speaking my native language) Anxiety, a student’s reported uneasiness about English (or internal state of concern or worry), was operationalized by items such as 31 (I get nervous about how to say things in English) and 32 (I worry that my English is not good enough) Stress, a student’s perception of external pressures, was operationalized by items such as 25 (I study more because my family wants me to) and 26 (I feel stressed because of my family’s expectations) Motivation was operationalized by items such as 20 (I am serious about my academic studies) and 21 (I am serious about my English language studies) The questionnaire allowed us to collect data necessary to confirm or reject our hypothesized model of the impact of language program type, ESL or EAP, on student engagement (academic and social) Analysis In order to test the relationships in the hypothesized model (Figure 2), we applied structural equation modeling (SEM) procedures, which allowed us to simultaneously assess multiple paths in the model, to explicitly incorporate error in the analysis, and to consider overall estimates of model fit to the data as part of the analytical process (Wu & Zumbo, 2008) As D€ ornyei (2007) points out, the “big advantage” of an SEM modeling approach is that it “also includes directional paths between the variables and not just information about how the variables hang together” (p 238) In other words, SEM allowed us to test cause-and-effect relationships among the variables, and because it is a 66 TESOL QUARTERLY connect with the new academic cultures they are encountering As Peacock (2001) points out, effective strategies can be taught, and “learners who can use a wide range of appropriate strategies have a greater degree of self-sufficiency and are developing as autonomous learners” (p 284) Whereas in this study the variable strategy use mediated the impact of language support programs on academic engagement, language use and goal-directed practices did not This suggests that course outcomes that attempt to help students identify goals, plan for their future careers, or generally increase their use of goal-directed practices may have no significant impact on their reported degree of academic engagement Further, ESL or EAP course outcomes which are directed at increasing students’ overall use of English (language use), for example, by using English with friends outside class, by talking with professors or TAs, not appear to impact students’ academic engagement However, it is important to point out that goal-directed practices did have an impact on students’ reported degree of social engagement In Table 4, we examine the result of program impact on the reported level of social engagement of the L2 students participating in the study Although chi-square was significant for the path EAP?Goal-Directed Practices?Social Engagement, the RMSEA = 067 indicated a good fit of the model to the data Further, the solution converged in two iterations For the path ESL?Language Use?Social Engagement, the chisquare was significant, but good model fit is suggested by the RMSEA = 053, and the solution converged in two iterations In both cases (ESL and EAP) the t-values were significant Even though both ESL and EAP programs had a direct effect on the degree to which the L2 students considered in this study were socially engaged (Table 4), the variables that mediated that engagement differed across the two program types Goal-directed practices mediated the effect of both ESL and EAP programs on students’ reported social engagement However, ESL program impact on social engagement was also mediated by increased language use As discussed earlier, understanding the mediating effects of program outcomes helps to explain why and how ESL and EAP programs impact social engagement Changes in students’ goal-directed practices brought about as a result of a participation in EAP and ESL courses, and changes in students’ communicative language use as a result of their ESL courses, affected their level of social engagement and satisfaction with their life in Canada This finding suggests the benefit of ESL activities, which emphasize the building of community, interaction, and spoken communication It also highlights the important role of the language teacher as a cultural interpreter (Kramsch, 1993), who IMPACT OF ENGLISH LANGUAGE PROGRAMS IN CANADIAN UNIVERSITIES 71 orients L2 students to the new academic and social cultures they encounter in undergraduate study by, for example, helping students define short- and long-term goals, understand strategic program choices, and recognize the role of planning and hard work in ensuring success When ESL and EAP teachers, acting as cultural interpreters, help support the development of new academic identities for L2 students in transition to study in Canadian universities, they address the sense of loss that can characterize new students’ initial experience in university (Scanlon et al., 2007) and scaffold students’ capacity to forge connections, which are so important to retention and academic success In sum, although the impact of both ESL and EAP programs on social engagement was mediated by the goal-directed practices the students developed as a course outcome, only ESL course impact was mediated by increased language use This suggests that the emphasis on spoken English, communication, and interaction had an impact on the degree to which students engaged socially Given that a distinguishing characteristic of ESL programs is their focus on developing a sense of community; creating connections; and improving spoken fluency, communication, and interaction, this is important evidence that, for the participants in this study, this focus made a difference in students’ social engagement with the new Canadian contexts they were encountering Research Question 3: How student characteristics (i.e., levels of anxiety; external stress caused by pressure from parents, finances, etc.; and motivation) moderate course outcomes? Does moderation differ in relation to program characteristics (ESL or EAP)? If so, how does it differ? Knowing where and how language program characteristics were related to (mediated by) course outcomes and their resulting impact on academic and social engagement was only half of the story Based on our hypothesized model, it was also important to better understand when or for whom (Wu & Zumbo, 2008) these effects were at play In order to explore the moderating variables that lessened or increased the impact of mediating variables on these L2 students’ academic and social engagement, we investigated whether the model was the same for students with high and low levels of anxiety, stress, and motivation We ran the SEM analysis again, with the same statistical parameters, to see if high or low levels of one of the moderating variables that we had identified would influence the impact of ESL and EAP programs on academic and social engagement The results of this analysis are summarized in Table (academic engagement) and Table (social engagement) In reading these tables, the column at the far right is 72 TESOL QUARTERLY IMPACT OF ENGLISH LANGUAGE PROGRAMS IN CANADIAN UNIVERSITIES 73 380.64 395.65 381.5 340.65 356.29 344.2 392.4 414.62 423.69 200.62 207.16 189.21 340.65 356.29 344.2 191.96 200.62 209.47 Stress Motivation Anxiety Stress Motivation Anxiety Stress Motivation Anxiety Stress Motivation Chi-squared paths not set to equal (a) Anxiety Stress Motivation Anxiety Stress Motivation Anxiety Moderating variables Significant moderating effect: *p < 05 EAP?Strategy use?AE EAP?Strategy use?AE EAP?Strategy use?AE EAP?Language use?AE EAP?Language use?AE EAP?Language use?AE EAP?Goal-directed practices?AE EAP?Goal-directed practices?AE EAP?Goal-directed practices?AE ESL?Strategy use?AE ESL?Strategy use?AE ESL?Strategy use?AE ESL?Language use?AE ESL?Language use?AE ESL?Language use?AE ESL?Goal-directed practices?AE ESL?Goal-directed practices?AE ESL?Goal-directed practices?AE Program type, mediating variable, and impact on academic engagement (AE) 91 91 91 91 91 135 135 135 91 160 160 160 160 160 135 135 135 160 DF, paths not equal 215.71 202.89 202.94 212.41 191.32 345.01 359.73 350.17 200.53 429.95 416.88 384.76 398.81 383.85 345.01 359.73 350.17 402.43 Chi-squared paths set to equal (b) 94 94 94 94 94 138 138 138 94 163 163 163 163 163 138 138 138 163 DF, paths equal 6.24 2.27 2.32 5.25 2.11 4.36 3.44 5.97 8.57 6.26 2.26 4.14 3.16 2.35 4.36 3.44 5.97 10.03 (c) = Chi-squared difference test of moderator (b) À (a) = (c) TABLE Overview of Moderating Effects on the Impact of Language Support Programs on Academic Engagement (N = 641) 3 3 3 3 3 3 3 3 3 DF 100 518 509 154 550 225 329 113 036* 100 520 247 368 503 225 329 113 018* p-value No No No No No No No No Yes No No No No No No No No Yes Is there a moderating effect? [yes or no] 74 TESOL QUARTERLY 203.35 176.09 193.64 147.78 150.89 157.58 133.69 179.85 Anxiety Stress Motivation Anxiety Stress Motivation Anxiety Motivation 414.43 Motivation 195.65 421.47 Stress Stress 369.63 395.16 344.24 337.18 348.09 316.49 384.7 Anxiety Stress Motivation Anxiety Stress Motivation Anxiety Moderating variables Chi-squared paths not set to equal (a) Significant moderating effect: *p < 05; **p < 01 EAP?Strategy use?SE EAP?Strategy use?SE EAP?Strategy use?SE EAP?Language use?SE EAP?Language use?SE EAP?Language use?SE EAP?Goal-directed practices?SE EAP?Goal-directed practices?SE EAP?Goal-directed practices?SE ESL?Strategy use?SE ESL?Strategy use?SE ESL?Strategy use?SE ESL?Language use?SE ESL?Language use?SE ESL?Language use?SE ESL?Goal-directed practices?SE ESL?Goal-directed practices?SE ESL?Goal-directed practices?SE Program type, mediating variable, and impact on social engagement (SE) 91 91 91 91 91 72 72 72 91 160 160 160 160 160 135 135 135 160 DF, paths not equal 212.6 201.27 179.38 227.65 160.41 154.62 158.04 148.22 186.43 425.3 416.55 373.43 402.86 356.92 341.26 349.56 330.17 390.31 Chi-squared paths set to equal (b) 94 94 94 94 94 75 75 75 94 163 163 163 163 163 138 138 138 163 DF, paths equal 9.25 5.62 3.29 34.01 12.63 3.73 46 14.53 6.58 10.87 4.08 3.8 7.7 12.68 4.08 1.47 13.68 5.61 (c) = Chi-squared difference test of moderator (b) À (a) = (c) TABLE Overview of Variables in Moderating the Impact of Language Support Programs on Social Engagement (N = 641) 3 3 3 3 3 3 3 3 3 DF 026* 123 349 000** 006** 292 928 002** 087 012* 253 284 053* 005** 253 689 003** 132 p- value Yes No No Yes Yes No No Yes No Yes No No Yes Yes No No Yes No Is there a moderating effect? [yes or no] key This column indicates whether each of the variables moderated the impact of program on engagement The three columns that precede the final column provide details of the chi-squared difference test, degrees of freedom (df), p-values, and significance, which is indicated by the presence of an asterisk Further details of the chi-squared difference test are found in the remaining columns of the tables With regard to the path EAP?Goal-Directed Practices?Academic Engagement (see Table 5), we found that a student’s level of anxiety acts as a significant (p < 05) moderator for the effect In other words, the effect was significantly different for groups reporting high and low levels of anxiety The greater the level of anxiety, the lower the impact of EAP on goal-directed practices (mediating variable) This suggests that L2 students with high levels of anxiety may be much less able to set short- and long-term goals, make program choices, plan, and so forth An L2 student who, because of a high level of anxiety, fails to engage in goal-directed practices to the same degree as his or her less anxious peers may also derive fewer benefits from the EAP course, which in turn lowers the impact of the EAP course on the student’s academic engagement Anxiety appears to moderate the impact of ESL programs on academic engagement as well: ESL?Goal-Directed Practices?Academic Engagement (p < 05) It would appear based on these results that high levels of anxiety (operationalized here by items which elicited students’ feelings of shyness, lack of confidence, worry, nervousness, etc.) undermine the impact of both EAP and ESL course support on academic engagement Interestingly, neither of the other student characteristics considered here, namely, stress (caused by external pressures from parents, finances, course demands, etc.) or motivation, acted as moderators, increasing or lessening the effect of language support programs on academic engagement Although there is a long history in the literature regarding the positive role of motivation in language learning (D€ ornyei, 2001), in this study reported levels of motivation did not have a significant impact on academic engagement However, stress and motivation appeared to play a significant role in moderating the impact of ESL and EAP courses on social engagement As summarized in Table 6, stress moderated the impact of EAP courses on social engagement There were significant differences (p < 01) in the reported levels of social engagement (i.e., satisfaction and interaction with study and life in Canada, connections with other students, a sense of community, belonging) across the high- and lowstress groups In other words, high stress caused by external pressures such as parents’ expectations, challenging finances, course demands, time limitations, and so forth appeared to undermine to a significant degree the impact of EAP courses on students’ development of IMPACT OF ENGLISH LANGUAGE PROGRAMS IN CANADIAN UNIVERSITIES 75 effective strategies for social engagement with the life and academic study cultures they were transitioning to This in turn undermined the impact of EAP courses on their social engagement However, stress was found to have a moderating effect on the development of these students’ skills and strategies only in EAP programs The same effect was not found in the analysis of ESL programs Indeed, motivation was a much more powerful moderating variable, with significant impact on all three mediating variables: strategy use (p < 01), English language use (p < 05), and goal-directed practices (p < 05) Whether a program had an EAP emphasis (e.g., academic language use, academic literacy and commensurate skills and strategy development) or an ESL emphasis (e.g., development of effective communication skills, improved speaking and interaction skills), motivation significantly moderated program impact on social engagement This study supports other research that has identified motivation as an important, significant predictor of success or achievement in language learning (D€ ornyei, 2001; Fox, 2009) CONCLUSION Overall, this study helped us develop a greater understanding of relationships between EAP and ESL program characteristics and L2 students’ academic and social engagement in Canadian universities We reported evidence that supported the hypothesized moderated mediation model proposed in Figure As part of our concluding remarks, we would first like to acknowledge the study’s limitations We did not draw a random sample, although the participants were arguably representative of the overall population of L2 learners currently taking English language courses in English-medium Canadian universities, nor did we include language programs offered at French-medium universities in Canada Second, given the large numbers of L2 students in ESL and EAP programs in Canadian universities, our sample size was relatively small Further, we did not study specific groups of L2 students, but examined instead a broad range of students in both ESL and EAP courses, because they are the predominant program types offered by Canadian universities We also recognize that our findings are based solely on students’ questionnaire responses and their perceptions of language support in their university study contexts Perceptions, although an informative source of data, are limited Further, because this was a SEM study, it is important to acknowledge that it is not definitive SEM “does not identify causation but only informs the researcher as to whether a hypothesized cause-effect relation is conceivable based on the total amount of 76 TESOL QUARTERLY data” (D€ ornyei, 2007, p 240) Thus, given these limitations, we must interpret the findings of this study with caution With these caveats in mind, we would like to highlight key findings from the study First and most important, this study provides empirical evidence that English language programs had a direct, positive, and significant effect on the academic and social engagement of the L2 students considered here Clearly, in the face of cutbacks and accountability agendas, such evidence is essential for the survival and growth of ESL and EAP programs in Canadian universities, because of their important (and in many cases essential) contribution to L2 students’ potential Increasingly, ESL and EAP programs are asked by university administrators to account for their impact on L2 students Given that a student’s level of academic and social engagement is significantly related to retention, academic success, and program completion (Fox, 2005; Scanlon et al., 2007), findings like those reported in this study should help make the case that ESL and EAP support often plays a critical role in a student’s ultimate academic success In other words, such language support programs indeed make a difference This study provides evidence that both ESL and EAP programs, which are characterized by different pedagogical approaches and/or activities, had a direct, positive impact on student engagement Whether the emphasis was on a narrower academic focus (as it was in the EAP programs considered here) or a broader communicative focus (as it was for ESL programs), there were direct effects on students’ academic and social engagement Second, it is interesting here to see the interaction between students’ personal characteristics and the impact of language support on their engagement This study begins to clarify the role of students’ personal characteristics (their levels of internal anxiety; the stress they feel from external pressures of family, finances, time, and so forth; and the important role of motivation) in lessening or strengthening the impact of language support on academic and social engagement This finding supports calls in the literature for more targeted instruction (Cheng et al., 2004; Fox, 2009) and systematic diagnostic assessment which is linked to relevant pedagogical activity (Fox, 2009; Fox & Hartwick, 2011) The better course providers understand the needs and strengths of their individual students with regard to both academic and social engagement, the greater the potential impact of language support Further, aligning course activity with students’ agendas will help increase course impact (see Cheng & Fox, 2008, for a full discussion) Finally, it is important to note that this study surveyed only official university language support programs (i.e., programs that were members of professional organizations such as TESL Canada and TESL Ontario or were described on university web pages) It did not survey IMPACT OF ENGLISH LANGUAGE PROGRAMS IN CANADIAN UNIVERSITIES 77 the many ad hoc programs that are becoming increasingly prevalent These programs arise because of a local need and tend to be staffed by teachers who work for less money (typically on temporary, shortterm contracts, with no possibility of full-time employment, and without work benefits such as holiday pay or health insurance) Typically, such teachers have more limited training, background, or expertise in language teaching Universities in Canada are also increasingly handing over responsibility for language support to private businesses outside the university, which also hire teachers on short-term contracts and often without the expertise that has characterized language teaching professionals working in university programs Such teachers may have relevant content knowledge (e.g., mathematics, science) and because they speak English, they are assumed to have the capability to teach English If this trend continues it may ultimately deprofessionalize English language teaching at the university level in Canada (see Fulcher, 2009, for an account of similar trends in the United Kingdom) One of the great strengths of language teaching which is situated within the university and offered by language teaching professionals is the ongoing interchange they occasion as participating members of the academic community, and the resulting academic richness that envelops ESL and EAP teaching occurring in the context of the university itself Language teaching outside the university does not benefit from, and cannot claim to develop, the connections that, as this study suggests, are essential to the academic and social engagement of L2 students and directly support their retention and academic success in university In sum, the findings of this study suggest that EAP and ESL programs make a significant difference in L2 students’ transition to a new academic study culture in university There is also a pressing need to systematically investigate and further clarify the relationship between language support program emphasis, students’ personal characteristics, and successful transition to university study ACKNOWLEDGMENTS This research was supported by the Social Sciences and Humanities Research Council of Canada We would like to thank Robert Berman, Christine Doe, Saira Fitzgerald, and Yeng Zheng for their assistance in this study THE AUTHORS Janna Fox is associate professor in applied linguistics and language studies at Carleton University, in Ottawa, Canada Her research focuses on assessment, teaching, 78 TESOL QUARTERLY and learning in culturally and linguistically diverse settings She holds a 3M National Teaching Fellowship in recognition of contributions to scholarship and leadership in Canadian higher education Liying Cheng is professor in the Faculty of Education at Queen’s University, in Kingston, Ontario, Canada Her primary research interests are the impact of largescale testing on instruction; the relationships between assessment and instruction; and the academic and professional acculturation of international and new immigrant students, workers, and professionals to Canada Bruno D Zumbo is professor of measurement, evaluation, and research methodology at the University of British Columbia His primary research interests are in multivariate analysis and psychometric methods Among his honors, he is a fellow of the American Educational Research Association and a recipient of both the Messick Award and the Killam Teaching award REFERENCES Al-Sharideh, K., & Goe, W (1998) Ethnic communities within the university: An examination of factors influencing the personal adjustment of international students Research in Higher Education, 39, 699–725 doi:10.1177/1475240 906065589 Banerjee, J., & Wall, D (2006) Assessing and reporting performances on pre-sessional courses: Developing a final assessment procedure Journal of English for Academic Purposes, 5(1), 50–69 doi:10.1016/j.jeap.2005.11.003 Berman, R (2002, May) Toward a common concept of English for academic purposes In L Cheng (Chair), Concepts and contexts in EAP: Preparing NNS of English for success in the academic community Symposium conducted at the Canadian Association of Applied Linguists, Congress of the Social Sciences and Humanities, Toronto, Canada Berman, R., & Cheng, L (2001) English academic language skills: Perceived difficulties by undergraduate and graduate students, and their academic achievement Canadian Journal of Applied Linguistics, 4, 25–40 doi:10.1177/ 1475240906065589 Byrne, B M (1994) Structural equation modeling with EQS and EQS/Windows: Basic concepts, applications and programming Thousand Oaks, CA: Sage Byrne, B M (1998) Structural equation modeling with LISREL, PRELIS, and SIMPLIS: Basic concepts, applications, and programming Mahwah, NJ: Lawrence Erlbaum Charmaz, K (2006) Constructing grounded theory: A practical guide through qualitative analysis London, England: Sage Cheng, L., & Fox, J (2008) Towards a better understanding of academic acculturation: Second language students in Canadian universities Canadian Modern Language Review, 65, 307–333 doi:10.1353/cml.0.0025 Cheng, L., Myles, J., & Curtis, A (2004) Targeting language support for nonnative English speaking graduate students at a Canadian university TESL Canada Journal, 22, 50–71 Retrieved from http://www.teslcanadajournal.ca/ Christopher, V (1993) Direct and indirect placement test scores as measures of language proficiency and predictors of academic success for ESL students (Unpublished master’s thesis) University of British Columbia, Vancouver, Canada Retrieved from http://circle.ubc.ca/bitstream/handle/2429/4886/ubc_1994-0053.pdf? sequence=1 IMPACT OF ENGLISH LANGUAGE PROGRAMS IN CANADIAN UNIVERSITIES 79 Cotton, F., & Conrow, F (1998) An investigation of the predictive validity of IELTS amongst a group of international students studying at the University of Tasmania English Language Testing System Research Reports, 1, 72–115 Retrieved from www.ielts.org/pdf/Vol1_Report4.pdf D€ ornyei, Z (2001) Teaching and researching motivation London, England: Pearson Education D€ ornyei, Z (2007) Research methods in applied linguistics Oxford, England: Oxford University Press Fox, J (2004) Test decisions over time: Tracking validity Language Testing, 21, 437–465 doi:10.1191/0265532204lt292oa Fox, J (2005) Rethinking second language admission requirements: Problems with language-residency criteria and the need for language assessment and support Language Assessment Quarterly, 2, 85–115 doi:10.1207/s1543431 1laq0202_1 Fox, J (2009) Moderating top-down policy impact and supporting EAP curricular renewal: Exploring the potential of diagnostic assessment Journal of English for Academic Purposes, 8, 26–42 doi:10.1016/j.jeap.2008.12.004 Fox, J., Cheng, L., Berman, R., Song, X., & Myles, J (2006) Costs and benefits: English for academic purposes instruction in Canadian universities In J Fox & C Doe (Eds.), Carleton papers in applied language studies (Vol 23, pp 1–108) Ottawa, Ontario, Canada: Carleton University Retrieved from http://www.carleton.ca/slals/ccms/wp-content/ccms-files/cpals-2006-Fox-et-al.pdf Fox, J., & Hartwick, P (2011) Taking a diagnostic turn: Reinventing the portfolio in EAP classrooms In D Tsagari & I Csepes (Eds.), Classroom-based language assessment (pp 47–62) Frankfurt, Germany: Peter Lang Freedman, A (1999) Beyond the text: Towards understanding the teaching and learning of genres TESOL Quarterly, 4, 764–767 doi:10.2307/3587890 Fulcher, G (2009) The commercialization of EAP provision in higher education In J C Alderson (Ed.), The politics of language education: Individuals and institutions (pp 125–146) London, England: Multilingual Matters Green, A (2007) Washback to learning outcomes: A comparative study of IELTS preparation and university pre-sessional language courses Assessment in Education, 14(1), 75–97 doi:10.1080/09695940701272880 Hansen, J (2000) Interactional conflicts among audience, purpose, and content knowledge in the acquisition of academic literacy in an EAP course Written Communication, 17, 27–52 doi:10.1177/0741088300017001002 Ivanic, R (1998) Writing and identity: The discoursal construction of identity in academic writing Amsterdam, the Netherlands: John Benjamins Ivanic, R (2006) Language, learning and identification In R Kiely, P ReaDickins, H Woodfield, & G Clibbon (Eds.), Language, culture and identity in applied linguistics (pp 7–29) London, England: British Association for Applied Linguistics Johns, A (1997) Text, role, and context: Developing academic literacies Cambridge, England: Cambridge University Press Johnson, P (1988) English language proficiency and academic performance of undergraduate international students TESOL Quarterly, 22, 164–168 doi:10 2307/3587070 Jordan, R (2004) English for academic purposes Cambridge, England: Cambridge University Press Kerstjens, M., & Nery, C (2000) Predictive validity in the IELTS test: A study of the relationship between IELTS scores and students’ subsequent academic 80 TESOL QUARTERLY performance English Language Testing System Research Reports, 3, 85–108 Retrieved from www.ielts.org/pdf/Vol3_Report4.pdf Kramsch, C (1993) Context and culture in language teaching Oxford, England: Oxford University Press Leki, I (1995) Coping strategies of ESL students in writing tasks across the curriculum TESOL Quarterly, 29, 235–260 doi:10.2307/3587624 Leki, I (2001) “A narrow thinking system”: Nonnative-English-speaking students in group projects across the curriculum TESOL Quarterly, 35, 39–67 doi:10 2307/3587859 Loehlin, J C (2004) Latent variable models: An introduction to factor, path, and structural analysis (2nd ed.) Mahwah, NJ: Lawrence Erlbaum Macrae, M (1997) The induction of international students to academic life in the United Kingdom In D McNamara & R Harris (Eds.), Overseas students in higher education: Issues in teaching and learning (pp 127–142) London, England: Routledge Moore, T., & Morton, J (2005) Dimensions of difference: A comparison of university writing and IELTS writing Journal of English for Academic Purposes, 4(1), 43–66 doi:10.1016/j.jeap.2004.02.001 Morita, N (2000) Discourse socialization through oral classroom activities in a TESL graduate program TESOL Quarterly, 34, 279–310 doi:10.2307/3587953 Muller, D., Judd, C M., & Yzerbyt, D Y (2005) When moderation is mediated and mediation is moderated Journal of Personality and Social Psychology, 89, 852– 863 doi:10.1037/0022-3514.89.6.852 Peacock, M (2001) Language learning strategies and EAP proficiency: Teacher views, student views, and test results In J Flowerdew, & M Peacock (Eds.), Research perspectives on English for academic purposes (pp 268–285) Cambridge, England: Cambridge University Press Pennycook, A (1996) Borrowing others’ words: Text, ownership, memory, and plagiarism In V Zamel & R Spack (Eds.), Negotiating academic literacies (pp 265–292) Mahwah, NJ: Lawrence Erlbaum Savignon, S (2002) Interpreting communicative language teaching New Haven, CT: Yale University Press Scanlon, L., Rowling, L., & Weber, Z (2007) “You don’t have like an identity you are just lost in a crowd”: Forming a student identity in the first-year transition to university Journal of Youth Studies, 10, 223–241 doi:10.1080/ 13676260600983684 Shih, S F., & Brown, C (2000) Taiwanese international students: Acculturation level and vocational identity Journal of Career Development, 27(1), 35–47 doi:10 1177/089484530002700103 Swales, J., Barks, D., Ostermann, A., & Simpson, R (2001) Between critique and accommodation: Reflections on an EAP course for masters of architecture students English for Specific Purposes, 20(Supplement 1), 439–458 doi:10.1016/ S0889-4906(01)00020-5 Wu, A D., & Zumbo, B (2008) Understanding and using mediators and moderators Social Indicators Research, 87, 367–392 doi:10.1007/s11205-007-9143-1 IMPACT OF ENGLISH LANGUAGE PROGRAMS IN CANADIAN UNIVERSITIES 81 APPENDIX SURVEY OF ENGLISH SUPPORT PROGRAMS FOR SECOND LANGUAGE (L2) STUDENTS IN CANADIAN UNIVERSITIES 82 TESOL QUARTERLY IMPACT OF ENGLISH LANGUAGE PROGRAMS IN CANADIAN UNIVERSITIES 83 84 TESOL QUARTERLY IMPACT OF ENGLISH LANGUAGE PROGRAMS IN CANADIAN UNIVERSITIES 85