ADVANCING QUANTITATIVE METHODS IN SECOND LANGUAGE RESEARCH By picking up where introductory texts have left off, Advancing Quantitative Methods in Second Language Research provides a “second course” on quantitative methods and enables second language researchers to both address questions currently posed in the field in new and dynamic ways and to address novel or more complex questions as well In line with the practical focus of the book, each chapter provides the conceptual motivation for and step-by-step guidance needed to carry out a relatively advanced, novel, and/or underused statistical technique Using readily available statistical software packages such as SPSS, the chapters walk the reader from conceptualization through to output and interpretation of a range of advanced statistical procedures such as bootstrapping, mixed effects modeling, cluster analysis, discriminant function analysis, and meta-analysis This practical hands-on volume equips researchers in applied linguistics and second language acquisition (SLA) with the necessary tools and knowledge to engage more fully with key issues and problems in SLA and to work toward expanding the statistical repertoire of the field Luke Plonsky (PhD, Michigan State University) is a faculty member in the Applied Linguistics program at Northern Arizona University His interests include SLA and research methods, and his publications in these and other areas have appeared in Annual Review of Applied Linguistics, Applied Linguistics, Language Learning, Modern Language Journal, and Studies in Second Language Acquisition, among other major journals and outlets He is also Associated Editor of Studies in Second Language Acquisition and Managing Editor of Foreign Language Annals SECOND LANGUAGE ACQUISITION RESEARCH SERIES Susan M Gass and Alison Mackey, Series Editors Monographs on Theoretical Issues: Schachter/Gass Second Language Classroom Research: Issues and Opportunities (1996) Birdsong Second Language Acquisition and the Critical Period Hypotheses (1999) Ohta Second Language Acquisition Processes in the Classroom: Learning Japanese (2001) Major Foreign Accent: Ontogeny and Phylogeny of Second Language Phonology (2001) VanPatten Processing Instruction: Theory, Research, and Commentary (2003) VanPatten/Williams/Rott/Overstreet Form-Meaning Connections in Second Language Acquisition (2004) Bardovi-Harlig/Hartford Interlanguage Pragmatics: Exploring Institutional Talk (2005) Dörnyei The Psychology of the Language Learner: Individual Differences in Second Language Acquisition (2005) Long Problems in SLA (2007) VanPatten/Williams Theories in Second Language Acquisition (2007) Ortega/Byrnes The Longitudinal Study of Advanced L2 Capacities (2008) Liceras/Zobl/Goodluck The Role of Formal Features in Second Language Acquisition (2008) Philp/Adams/Iwashita Peer Interaction and Second Language Learning (2013) VanPatten/Williams Theories in Second Language Acquisition, Second Edition (2014) Leow Explicit Learning in the L2 Classroom (2015) Dörnyei/Ryan The Psychology of the Language Learner—Revisited (2015) Monographs on Research Methodology: Tarone/Gass/Cohen Research Methodology in Second Language Acquisition (1994) Yule Referential Communication Tasks (1997) Gass/Mackey Stimulated Recall Methodology in Second Language Research (2000) Markee Conversation Analysis (2000) Gass/Mackey Data Elicitation for Second and Foreign Language Research (2007) Duff Case Study Research in Applied Linguistics (2007) McDonough/Trofimovich Using Priming Methods in Second Language Research (2008) Dörnyei/Taguchi Questionnaires in Second Language Research: Construction, Administration, and Processing, Second Edition (2009) Bowles The Think-Aloud Controversy in Second Language Research (2010) Jiang Conducting Reaction Time Research for Second Language Studies (2011) Barkhuizen/Benson/Chik Narrative Inquiry in Language Teaching and Learning Research (2013) Jegerski/VanPatten Research Methods in Second Language Psycholinguistics (2013) Larson-Hall A Guide to Doing Statistics in Second Language Research Using SPSS and R, Second Edition (2015) Plonsky Advancing Quantitative Methods in Second Language Research (2015) Of Related Interest: Gass Input, Interaction, and the Second Language Learner (1997) Gass/Sorace/Selinker Second Language Learning Data Analysis, Second Edition (1998) Mackey/Gass Second Language Research: Methodology and Design (2005) Gass with Behney & Plonsky Second Language Acquisition: An Introductory Course, Fourth Edition (2013) ADVANCING QUANTITATIVE METHODS IN SECOND LANGUAGE RESEARCH Edited by Luke Plonsky NORTHERN ARIZONA UNIVERSITY First published 2015 by Routledge 711 Third Avenue, New York, NY 10017 and by Routledge Park Square, Milton Park, Abingdon, Oxon, OX14 4RN Routledge is an imprint of the Taylor & Francis Group, an informa business © 2015 Taylor & Francis The right of Luke Plonsky to be identified as the author of the editorial material, and of the authors for their individual chapters, has been asserted in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988 All rights reserved No part of this book may be reprinted or reproduced or utilised in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers Trademark notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe Library of Congress Cataloging-in-Publication Data Plonsky, Luke Advancing quantitative methods in second language research / Luke Plonsky, Northern Arizona University pages cm — (Second Language Acquisition Research Series) Includes bibliographical references and index 1. Second language acquisition—Resesarch. 2. Second language acquisition—Data processing. 3. Language and languages—Study and teaching—Research. 4. Language acquisition—Research. 5. Language acquisition—Data processing. 6. Quantitative research. 7. Multilingual computing. 8. Computational linguistics. I. Title P118.2.P65 2015 401'.93—dc23 2014048744 ISBN: 978-0-415-71833-2 (hbk) ISBN: 978-0-415-71834-9 (pbk) ISBN: 978-1-315-87090-8 (ebk) Typeset in Bembo by Apex CoVantage, LLC For Pamela This page intentionally left blank CONTENTS List of Illustrations xi Acknowledgments xvii xix List of Contributors PART I Introduction 1 1 Introduction Luke Plonsky Why Bother Learning Advanced Quantitative Methods in L2 Research? 9 James Dean Brown PART II Enhancing Existing Quantitative Methods Statistical Power, p Values, Descriptive Statistics, and Effect Sizes: A “Back-to-Basics” Approach to Advancing Quantitative Methods in L2 Research Luke Plonsky A Practical Guide to Bootstrapping Descriptive Statistics, Correlations, t Tests, and ANOVAs Geoffrey T LaFlair, Jesse Egbert, and Luke Plonsky 21 23 46 Bayesian Informative Hypothesis Testing 343 FIGURE 14.9 Comparison of Means Bayesian analysis output 5.25 times more likely (42.8/8.14) than the “collapse down” alternative hypothesis, H4 The PMP (see Figure 14.9) estimates H1 to be unambiguously the most probable with a PMP of 82, given the data The posterior model probability for each hypothesis is the Bayes factor for that particular hypothesis divided by the sum of the Bayes factors for all of the hypotheses tested The Bayes factor and the PMP estimates suggest that the ordering of mean difficulties predicted by the item design specification and construction system is corroborated by the empirical data based on large samples of test takers We see that the ordering of mean difficulties predicted by the item design specification and construction system is corroborated by the empirical data based on large samples of test takers The Bayesian confirmatory approach provides a useful diagnostic for identifying languages and modalities in the test development framework that might be in need of further development and moderation Discussion Confirmatory analyses were conducted to both illustrate the potential for an ANOVA software program, Comparison of Means (Kuiper and Hoijtink, 2010), 344 Beth Mackey and Steven J. Ross and to consider the strengths and weaknesses of an item-based approach to defining proficiency levels in a testing program Here we see that the predicted hierarchy of mean item difficulties matches the predictions from Level to Level Were we to find that the data support a different hypothesis, it would suggest that the test development process is imperfect, possibly stemming from the test developers’ inability to accurately predict item difficulty at adjacent levels An implication of the confirmatory analysis for an item-based system of defining proficiency may well be that alternative methods of standard setting are justified An alternative to the system outlined here would be a modified Angoff standard setting approach uninformed by item writers’ intention to write and tweak items to particular levels of difficulty Such a system would require standard-setting panels to decide, item by item, whether test takers would be likely to succeed at each level of the proficiency hierarchy Whether subjective item classification by item designers, moderation committees, or standard-setting panels will ultimately be corroborated by Bayesian confirmatory analyses is an issue that should be examined empirically in future research Limitations and Conclusion Bayesian confirmatory analysis of variance provides a robust alternative to conventional ANOVA The BMS tool used here is a new analysis technique in language testing As framework-based test development gains momentum around the world, there is a pressing need to examine the evidence that tests can be written to frameworks such as the Common European Framework of Reference (CEFR) and those developed by the Interagency Language Roundtable (ILR) and American Council on the Teaching of Foreign Languages (ACTFL) The Bayesian approach may offer a cost-effective intermediate step between test development and expensive and subjective standard setting methods The confirmatory approach outlined in this chapter is likely to provide a useful analytic tool for testing framework-generated hypotheses about language proficiency, as well as a viable alternative to null hypothesis testing of group mean differences in general Tools and Resources Additional explanatory notes and sample data are available with the Comparison of Means tool: http://www.uu.nl/staff/rmkuiper Further Reading Hoijtink, H., Klugkist, I., and Boelen, P (2008) Bayesian evaluation of informative hypotheses New York: Springer Kruschke, J (2011) Doing Bayesian data analysis: A tutorial with R and BUGS New York: Academic Press Bayesian Informative Hypothesis Testing 345 Discussion Questions Discuss when a null hypothesis test for an ANOVA design is superfluous and when a Bayesian approach might be more appropriate Discuss what makes a hypothesis “informative.” Identify areas of applied linguistics that are amenable to ordered hypotheses Discuss how meta-analysis results can inform informed hypothesis testing Sketch out a study on corrective feedback on L2 writing (or another area of L2 research) that predicts an order for different condition or treatment How and why might the approach illustrated in this chapter be used to examine such a set of predictions? References Hoijtink, H (2012) Informative hypotheses New York: Chapman Hall/CRC Hoijtink, H., Klugkist, I., and Boelen, P (2008) Bayesian evaluation of informative hypotheses New York: Springer Kruschke, J (2011) Doing Bayesian data analysis: A tutorial with R and BUGS New York: Academic Press Kuiper, R. M., & Hoijtink, H (2010) Comparisons of means using exploratory and confirmatory approaches Psychological Methods, 15(1), 69–86 doi:10.1037/a0018720 Lunn, D., Jackson, C., Best, N., Thomas, A., and Speigelhaler, D (2012) The BUGS book: A practical introduction to Bayesian analysis London: Taylor & Francis Ntzouflas, I (2009) Bayesian modeling using WinBUGS New York: John Wiley and Sons This page intentionally left blank INDEX ACTFL see American Council on the Teaching of Foreign Languages agglomerative hierarchical clustering 246 agglomeration schedule 248, 256 – 7 Aiken 133 – 6, 156, 158 Aldenderfer 252, 255, 257, 273 Algina 75 – 6 Allen 196, 211 Allison 138 – 9, 156, 158 alpha 39, 49, 110 alternative hypothesis 333 – 4, 336, 343 American Council for Teachers of Foreign Languages (ACTFL) 331, 344 AMOS 184, 215, 219, 222, 225, 235, 238 – 9; importing data in 226, 234 ANCOVA 132, 143 Anderson 187, 210 Andrés 162, 180 Andrich 277, 292 – 3, 303 Angoff 344 analysis of variance see ANOVA ANOVA 3, 35; 172, 174 – 5; Bayesian approach, compared to 329 – 30, 335, 343 – 4; cluster analysis and 261, 261, 269; discriminant analysis, relationship to 306, 312, 315; factor analysis and 183; mixed effects, compared to 160 – 2, 168; MRA in 132 – 3, 143, 147 – 8, 153; one-way 49, 68 – 9, 261, 261, 329 Anscombe 104 aptitude 35, 244, Arbuckle 238, 241 Asención-Delaney 183, 210 Asher 27 – 8, 45, 117, 128 assumptions 29; in Bayesian analysis 331 – 3; in bootstrapping 47, 49 – 50, 75; in confirmatory factor analysis 184; in discriminant analysis 307, 309 – 10, 313, 315, 321; in exploratory factor analysis 185, 187, 197; in meta-analysis 113 – 14, 135 – 6, 137; in mixed effects 160, 162; in multiple regression 140 – 1; of normality 170; in SEM 220, 236; statistical 3, 10, 16 – 19, 29, 177, 313, 321, 326 attrition 162 August 287, 290, 304 average 36; 39; 54; 114 – 15; 194; 202; 252; 312 Baayen 161, 172 – 4, 178 – 9 Baba 274 Babaii 5, Banks 111, 126 bar charts 79 – 80, 87 – 92, 102 Barkhuizen 299, 303 Barley 158 Barr 171, 178 – 9 Bartlett’s Test of Sphericity 188, 205 Bates 161, 164, 172 – 4, 175, 178 – 9, 181 348 Index Bayes factors 342 – 3 Bayesian analysis 335, 343; ANOVA, compared to 329 – 30, 335, 343 – 4; output 343; software for Bayesian model selection (BMS) 330, 334, 338 – 9, 344 Beasley 46 Becker 108, 125 Bentler 223, 238, 241 Bernhardt 134, 158 Best 329, 341, 345 between-groups design 112, 165, 171 bias: analysis of 299; in bootstrapping 60 – 1, 63 – 6, 69 – 73; correction of 51, 53, 54, 57; effect size 32, 113; positive 147; publication 38, 115 – 17, 119 – 21 Biber 245, 247 – 8, 269, 273, 274 Biggs 118, 125 Bikowski 246, 274, 274 Bithell 51, 76 Black 187, 210 Blashfield 252, 255, 257, 273 Boelen 333, 344, 345 Bonferroni 20 Boomsma 224, 241 Bond 294, 302 Borenstein 115, 122, 125, 127 bootstrapping 6, 11, 30, 46 – 9, 67, 69; BCa 51, 53, 54, 58; ANOVA 49; considerations in 50; confidence intervals 51; correlations 49; correlation coefficient 60 – 2; diagnostics 50; F statistic 67; mean differences 64; nonparametric 74; pairwise comparisons 49, 67; procedure 53, 56 – 7, 60; replications; resampling/sampling method 50; software for 7, 73 – 4; t-test 63, 65 Bosker 160, 181 Bovaird 161, 180 box plot 12, 78, 90 – 3, 94 Box’s M 312, 316, 321 Boyle 162, 179 Brannick 111, 126 Brown, D 108 – 9, 123, 127 Brown, J. D 3, 6, 10 – 11, 16, 308 Brown, T 209 Buchner 29, 44 Bryk 160, 181 Byrne 162, 180, 215, 234, 238 – 9, 241 Canty 51, 56, 76 Carlo 287, 290, 304 Carpenter 51, 76 case study 255, 261 categorical variables 14, 87, 94; in cluster analysis 243, 261, 268; in MRA 132 category characteristic curve 292 Cattell 194, 210 CEFR 305 – 6, 331, 344 Chan, D. K.-S 113, 125 Chan, W 46, 76 charts 78 – 80; 82, 86 – 92, 97 – 9 Chen 168, 179 centroid method 252 Chernick 50, 51, 76 Cheung 113, 125 chi-square test: Bartlett’s Test 188; chi-square difference test 224; in discriminant analysis 231; multivariate outliers 137; model fit 175, 223; in SEM 224, 229, 234 Clark 83, 104, 161, 180 classical hypothesis testing 329 classical test theory (CTT) 276, 303 – 4 classification accuracy 313, 326, 334 cluster analysis 6, 14, 243 – 5, 268; discriminant analysis, compared with 307, 316; dendogram 255; factor analysis, compared with 194, 196; hierarchical 246 – 7, 258; k-means 246; procedure 246 – 7, 250, 253, 258 – 9, 261, 263 – 5, 266; SEM compared with 223; software for Collentine 183, 210 collinearity 138, 140; reducing 168 Cohen, J 5, 29, 36, 117, 124, 125, 133 – 6, 156, 158 Cohen, P 133 – 6, 156, 158 Cohen’s d 26 – 7; calculating 31 – 2, 38; interpreting 38; weighting 119; see effect sizes Cohen’s kappa 112 Cohn 108, 125 Collins 162, 180 comparative fit index (CFI) 223, 228 – 31, 233, 237 comparison group 10, 28 comparison of means (software) 334 – 8, 341, 343 communalities 187 – 8, 190, 193, 209 component 14, 101 – 2, 132, 182 – 5, 187, 193, 197, 199, 200 – 1, 211 communality 190, 209 Comrey 187, 194, 199, 203, 205, 210 confidence intervals (CIs): calculating 32 – 3, 39; interpreting 40; reporting 10, 23; statistical significance, test of 40 Index 349 confirmatory Bayesian model selection (BMS) 330 confirmatory factor analysis (CFA) 14, 183, 209, 216 construct 3, 13, 203, 209, 294; theoretical 197, 215, 221; underlying 183, 192, 207, 213 – 14, 216 – 17, 290; validity 330 – 1 continuous variable: in discriminant analysis 14; displaying visually 87; in mixed models 162; in SEM 220 control group 51; in Bayesian analysis 329; in cluster analysis 243 control variables 13, 174 convergence errors 177 Conway 185, 210 Cooper 109, 111, 122, 125 Cordray 111, 126 corpus 117, 183, 305; in discriminant analysis 307 corpus analysis 273, 307, 327, 328 correlation 141; bivariate 131, 137 – 8; 143, 213, 315, 321; bootstrapped 60; canonical 317; discriminant analysis, in 306, 315, 321, 323; factor analysis, in 198; factor loading 209; high 177, 310; low 188; matrix 155, 188, 192, 205, 220; multivariate 131, 146; Pearson 253, 313; random 176; sample size 187; SEM, compared to 213; SEM, in 221, 224, 231, 237 – 8; semi-partial 13; software for 7; statistics 131; underlying 182 – 3; within-groups 313 correlation coefficients: assumption checking, part of 188; bivariate 60, 131, 137 – 8, 213, 315; 321; bootstrapping 60 – 2; calculating 33, 34; EFA, in 185; effect size, as 117; multivariate 131 correlation matrix 188, 192, 205, 220, 238, 240 Costello 194, 197, 210 covariance matrix 220, 222 – 3, 226 – 8 covariance structure analysis 213, 241 covariates: MRA, in 149; mixed effects, in 162 Crawley 51, 76, 271 Cronbach’s alpha 208 crossed random effects 161, 179, 181 cross-sectional design 163 Csizér 244, 273 Csomay 245, 274 Cumming 117, 227, 274 Cunnings 6, 15, 24, 44, 178, 180 Curran 220, 242 Dalton, D. R 107, 125 Dalton, C. M 107, 125 Daniel 80, 105 data: screening 16 – 19; sharing 112 – 13; MRA, in 136 – 7; preparation 220; reduction 13, 80, 220; requirements 228, 238, 276, 279; SEM, in 225; transformations 74, 137, 252; visualization 103, 166 Davidson 161, 172 – 4, 178 – 9 Davison 46 – 7, 50 – 1, 56, 75, 76 default graphic 99, 101 De Glooper 214, 219, 242 degrees of freedom 114, 137, 173 – 4, 223 – 4, 228, 230, 233, 241 Delaney 109, 113, 126 delayed posttest 110 Deering 75 – 6 delta 198, 292, 339 dependent variable 6, 13 – 14, 35, 40; MRA, in 131, 139, 145; mixed effects, in 159, 162, 164 – 6, 168; cluster analysis, in 243, 244, 261; discriminant analysis, in 305 – 6, 308, 310; SEM, in 214, 219 determinant 188 – 9, 192, 316 dichotomous variable 132, 277 – 8, 296 DiCiccio 51, 75 discrete variables 14, 87, 214 discrim see discriminant function analysis discriminant function analysis 305; procedure 308, 312 – 13; results 323 DiStefano 220, 241 Dixon 168, 178 divisive hierarchical clustering 246 Dörnyei 184, 211, 244, 273 dot plot 94 dummy variable 132, 142 Durbin-Watson statistic 144, 147 152 EAP (English for academic purposes) 245 Eato 196, 212 Eckes 245, 252, 274 effect sizes: 27, 125; calculating 31 – 6; Cohen’s d 26, 27 – 8, 31, 106, 112 – 13, 117 – 19; confidence intervals for 32, 33 – 5, 115 – 16; correlations as 31, 32, 106; eta-squared 27, 31; Hedges g 32; interpreting 37 – 8, 45, 118, 123, 127; Kramer’s V 36; large 37, 117 – 18; Odds ratio 36; phi 36; R2 5 – 6, 35; reporting 36 – 7; small 27, 29, 117, 119; weighting 113, 122 Efron 46, 50, 51, 75 350 Index Egbert 24, 30, 33, 44, 48, 65, 76 – 7, 113, 127, 162, 181 eigenvalue 193 – 7, 205, 207 – 9, 317 Eiting 222, 242 Elder 299, 303 Ellis, N. C 36 Ellis, R 107 Enders 168, 179, 220, 241 Eouanzoui 274 EQS 184, 238 – 9, 241 Erdfelder 29, 44 Erdosy 274 error: free 219; variance 113 – 15, 185, 222 error sum of squares 252 eta squared 5, 27, 35, 49; ANOVA, in 132 – 3; calculating 35 Excel 7, 31, 39, 99, 112 – 3, 122, 279 – 80 experimental design 39, 160, 162 explanatory variable 154 exploratory factor analysis (EFA) 8, 187 – 8, 190, 196, 203, 205 – 12, 216, 223, 242 Faca 196, 212 Facets 7, 278 – 9, 283 – 301 Fabringar 185, 192, 209, 210 factor analysis 6, 12 – 13, 182 – 4, 209; cluster analysis, compared to 244 – 5, 247, 255, 258; conducting 187, 190 – 4, 197; confirmatory 209; discriminant analysis, compared to 306, 316 – 17; exploratory 185, 209; factor extraction, in 190, 193, 200 – 1; factor loadings, in 199; principal axis factoring 184, 193, 208; principal components analysis 182, 184 – 5, 187, 193, 294 – 5; reporting 205; rotation 197 – 9, 203; software for 7; structural equation modeling, compared to 216, 219 factor loadings: matrix 197 – 8 factor rotation 197 – 9, 203 factor scores 13, 186, 201 – 3, 244, 261 Faul 29, 44 Fern 118, 125 Ferris 246, 274 Few 78 – 80, 103, 105 Fidell 131 – 2, 137 – 8, 140 – 1, 143, 157 – 8, 183, 185, 187, 192, 211 Field 156, 178, 180, 182 – 5, 187, 188, 190, 193 – 4, 197 – 9, 201 – 3, 205, 209 – 10 Finch 220, 242 Finegan 245, 273 Finney 220, 241 Fisher 80, 105 fit indices 223, 228 – 9, 234, 240; NFI 233; NNFI 233; RMSEA 223, 228; (S)RMR 233; Tucker-Lewis 223 fit statistics 233, 287, 289, 294, 296, 301; item fit 289 – 90; person fit 287 – 8 Fitzmaurice 109, 113, 126 fixed effects 114 – 15, 159, 166, 168 – 77, 179; parameters 168, 173 Ford 185, 187, 196, 205, 210 forest plot 79, 115 – 16 Forstmeier 171, 181 Fox 164, 241, 180, 238, 294, 302 frequency: cases, of 89; data 36; linguistic features, of 245, 325; subjects, of 88 funnel plot 115 – 17, 119 – 20, 124 fusion coefficient 196, 255, 257 – 8, 276 gain scores 325 gamma 74 Ganschow 244, 274 Gass 3, 5, 7 – 8, 29, 37, 46 – 7, 76, 77, 110, 118, 127, 160, 181, 182, 211 Gelman 162, 180 General Linear Model (GLM) 5, 35 – 6, 38, 133, 306 generalizability 77, 127, 242 Gibbs sampler 341 Gillespie 168, 180 Glass 122, 125 Gleser 113, 125 Glorfeld 196, 210 Goh 207 – 8, 128 Goldstein 160, 162, 180 Gonulal 7, 8, 44, 182, 187 – 8, 194, 196 – 7, 205 Goo 108, 118, 126 goodness of fit statistics 228, 230, 233 Gorsuch 183, 185, 187, 194, 210 Götz 211, 244, 246 grammaticality judgment task (GJT) 110, 162 – 3 Granena 35, 44 Grant 246, 274 graphics 78 – 80, 83, 87, 90, 103 – 4; guidelines for 80, 83, 86 – 7 Gray 247 – 8, 269, 270, 273 Green 157, 288, 302 – 3 Gridlines 82, 87 – 8 Gries 178, 180, 245, 252, 271, 274, 307, 328 Gu 214, 219, 225, 235 – 41 Hair 187, 210 Hancock 214, 219, 220, 221, 224, 239, 241 Index 351 Hansson 162, 178, 180 Harrington 184, 210 Harshman 196, 211 Harzing 109, 125 Hashemi 5, Hattie 118, 125 Hayes, E 272 Hayes, H. K 80 – 1 Hayes, T 114 Hayton 196, 211 Hedges 114, 115, 117, 122, 125, 125 Henson 185, 201, 203, 211 Herrington 24, 30, 44, 46, 48, 76, 77, 78, 105 hierarchical: cluster analysis 246 – 7, 258; regression analysis 141, 149 – 50, 152, 154, 158 Higgins 113, 115, 122, 125, 128 Hill 162, 180 Hinkley 46 – 7, 50, 51, 56, 75, 76 Hiromori 244, 268, 274 histogram 17, 62, 66 – 8, 78, 87 – 8, 91 Hobbs 109, 113, 126 Hoijtink 330, 334, 336, 338, 345 Hoffman 161, 180 homogeneity: effect size, of 114; variance, of 18, 51, 309, 312, 316 homoscedasticity 141, 162 Hong 185, 187, 211 Howell 133, 157, 158 Hu 223, 241 Huang 119, 126 Huff 5, Huffcutt 185, 210 Hulstijn 214, 219, 242 Hunter 113, 122, 125, 127 Hyde 109, 113, 126 hypothesis testing 49, 224, 266; Bayesian Informative 329, 333 – 4, 338 – 9; null 15, 223, 329 – 30, 336, 344 individual differences 244 infit mean square 290, 299 informative hypothesis 329 In’nami 107, 109, 123, 125, 239, 241 instruments 112, 208, 217, 279, 294, 300 interaction: between raters 299; effects 10, 12; mixed effects model, in 160, 166, 168 – 9, 171, 173; statistical 101 Interagency Language Roundtable (ILR) 331, 344 intercept 131, 169 – 73, 177 interquartile range 40 – 1 interval scale 10, 287, 308 interview 224, 325 intraclass correlation 112 Iramaneerat 301 IRIS database 112 Isoda 244, 268, 274 item analysis 15, 183, 331 item development 334 item/person map 285 item response theory (IRT) 79, 275, 335 item separation 290 iteration 48, 198, 222, 341 Iberri-Shea 162, 180 IBM SPSS see SPSS Immer 80 – 1 imputation: data, of 162, 220 incremental fit index (IFI) 228, 230, 233 indicator 115, 159, 217 inferential statistics 79, 224, 276, 306 independence: observation, of 309, 144, 160; model 228, 230, 233, 234 independent variables 3, 14, 118; discriminant analysis, in 306, 310; mixed effects, in 161, 166, 173; multiple regression, in 131, 138 – 9; structural equation modeling, in 214, 219, 232 Kaiser 194, 211 Kaiser-Meyer-Olkin (KMO) 187 Kaiser’s criterion 194, 196 Kamil 134, 158 Kang 272, 274 Kantor 274 Kenyon 287, 290, 304 Kepes 111, 126 Keselman 75 – 6 Kirk 126 Klass 78, 82, 91, 105 Kline 118, 126, 183, 185, 211 Klugkist 333, 344, 345 Koizumi 107, 109, 123, 125, 239, 241 Jackson 329, 341, 345 Jaeger 162, 168, 178, 180 James 274 Jang 108, 112 – 13, 123, 126 Jarvis 246, 274 Jeon 6, 13, 14, 35, 113, 125, 158 Jiang 158 Jin 155, 158 Johnson 108, 115, 126, 127 Joiner 269, 274 Joliffe’s criterion 194 Jöreskog 213 – 14, 225, 228, 238, 242 Josefsson 162, 178, 180 Jowett 109, 113, 126 352 Index Kojic-Sabo 244, 246, 274 Kolmogorov-Smirnov 75 Komsta 56, 76 Kosslyn 78, 82, 105 Knoch 275, 299, 303, 304 Kruschke 329, 344 Kuiper 330, 334, 336, 338, 345 Kunnan 239, 242 kurtosis 17, 40 – 1, 57, 220 lab-based research 123 Lackey 187, 193, 199, 203, 205 – 6, 209, 211 LaFlair 24, 30, 44, 48, 65, 76 – 7, 113, 127, 162, 181 Lagrange multiplier 224 Lang 29, 44 language-as-fixed-effect fallacy 187 language testing 35, 127, 235, 344 large samples 49, 343; difficulty obtaining 30 Larson-Hall 4, 5, 7, 24, 30, 42, 43, 44, 46, 47, 48, 49, 75, 76, 77, 82, 90, 97, 105, 157, 178, 180, 205, 211 latent variable 183, 213 – 17, 219, 221 – 2, 225 – 9, 231 – 3 Lavolette 183, 188, 190, 194 – 7, 200 – 1, 203, 205, 209 – 10, 211 Lazaraton 47 – 8, 77, 160, 180 Le 114, 127 Lee, H. B 187, 194, 199, 203, 205 Lee, J 108, 112 – 13, 123, 126, 272 – 4 Lee, S.-K 119, 126 Lee, W. C 46 – 7 Leung 245, 252, 274 Levy 171, 173 – 4, 179 Li 107, 108, 118, 120, 124, 126 Lightbown 244, 246, 274 Likert scale 110, 183, 187, 247, 278 Linacre 278 – 9, 287, 290, 294, 301, 303, 304 Linck 178, 180 linearity 140, 307 linear regression 33 – 5, 131, 137 – 9, 144 – 5, 157 line graph 12, 78 – 9, 92, 94, 101 – 2 linguistic variables 240, 245, 262 linkage: between-groups 250; withingroups 250 Lipsey 110 – 12, 122, 126 LISREL 7, 184, 196, 213, 215, 219, 222, 225, 227 – 31, 234, 238 – 41 listwise deletion 61, 198, 220 Littre 163, 163, 178, 180 Lix 75 – 6 Ljungberg 162, 178, 180 Locker 161, 180 Loewen 182, 183, 188, 190, 194 – 7, 200 – 1, 203, 205, 209 – 10, 211 logit scale 176, 277, 285, 287, 289, 296 logit scores 10, 166 log-likelihood ratio 171 loglinear modeling 14 longitudinal data analysis 159 longitudinal design 159 logistic regression 14, 36, 132, 307, 309, 327 logit 10, 162, 166, 170, 176, 179, 277, 285, 287, 289 – 90, 296, 331, 333, 337; difficulty 333; mixed effects 162, 179; transformation 166, 170, 176 loglinear modeling 14 Louguit 287, 290, 304 Lunn329, 341, 345 Lyster 108, 118, 126 MacCallum 185, 187, 196, 205, 210, 211 McCloy 108, 127 McDaniel 111, 126 McGill 48, 77 Mackey 44, 108, 118, 126, 307 McManus 109, 113, 126 McNamara 275, 276, 288 – 9, 299, 304 Magnuson 168, 178 Mahalanobis distance 137 – 9 main effects 168, 173, 175, 178 Mak 155, 158 Malabonga 287, 290, 304 manifest variables 214 Mann-Whitney U test 47 MANOVA 306, 308 – 10, 312, 323, 326 many-faceted Rasch model 299 Marcoulides 239, 242 Mareschall 207 – 8, 128 Markov-Chain Monte-Carlo (MCMC) 341 Masters 278, 304 mathematical transformations 17 matrix algebra 17 – 18 maximal random effects structure 173, 176 maximum likelihood 170, 175, 184 – 5, 193, 222, 231, 301 mean square 148, 153, 288; infit 290, 299; root 223, 228 – 33 measurement error 219, 240 measurement model 215, 217, 221 Index 353 median 41, 55, 90 – 1, 112 – 13, 118, 165, 188, 306; clustering 252 – 3 meta-analysis 12, 27, 30, 36 – 7, 42, 106 – 28, 121; analysis 112 – 13; benefits of 106 – 7; coding 110, 111; data analysis 112 – 15; data collection/coding 110 – 12; defining the domain of 108; examples of 27, 36, 108 – 10, 120; forest plot 115 – 16; funnel plot 115 – 17; history of 107, 122; interpreting results 117 – 19; interrater reliability for coding 112; method 114; models 114 – 15; moderators 108, 114, 115, 119, 120 – 1, 123; publication bias 115 – 17, 119, 120, 121, 122, 124; Q-test 114; results 345; searching for primary studies 108 – 9; software for 112, 120, 121 – 2; weighting effect sizes 113 – 14 methodological reform 4 – 5, 24 Meunier 163, 163, 178, 180 Minimum Fit Function Chi-Square 228, 230, 233 Ministeps 301 Mirman 168, 178, 180 misfit 49, 221, 224, 288 – 91, 294, 299 missing data 15, 61, 113, 142, 143, 162, 198, 220, 276, 279; imputation of 162, 220 Mitchell 168, 179 mixed effects 6, 15, 159 – 70, 173, 175, 178 mixed methods Mizumoto 7, 8, 43, 184, 211 model building: competing or concurrent models 214, 217 – 18, 221; measurement model 215, 217, 221; model specification 222 – 3, 225 – 6, 231, 339; structural model 215 – 16, 221, 240 model comparison 174 – 5 model overfitting 221, 238, 289, 303, 310 moderator variables 13, 114, 123 modification indices: Lagrange Multiplier test 224; Wald test 224 Molenaar 219, 242 moments: distribution, of a 57 – 8; package 56 Monroe 118, 125 Morris 112, 126 Mueller 214, 219, 221, 224, 239, 241 multicollinearity 13, 137 – 40, 188, 192, 203, 309 – 10, 313, 316, 321; software for multiple comparisons 9, 11, 69 multiple regression 5 – 7, 10, 13 – 16; 35, 131; hierarchical 13, 224 multivariate analysis of variance see MANOVA multivariate normality 17, 140, 220, 228, 309; outliers 17 – 18, 137 multivariate statistics 3 – 4, 29 multiway analysis 14 Muthén, B O 236, 238 Muthén, L. K 236, 238 NAEP State Comparisons Tool from the National Center for Educational Statistics 84 – 5, 87, 105 Nakata 244, 274 Nassaji 49, 77, 307, 326, 328 National Assessment of Educational Progress (NAEP) 83 nested random effect 160 Newman 245, 252, 274 Nicol 82, 105 Nilsson 162, 178, 180 nominal variable 10, 305, 310 nonnormality 6, 11, 29, 46, 48 non-parametric statistics 74, 178 normal distribution 44, 62, 65, 67, 162, 166, 288; Satorra-Bentler’s correction 220 normality 11, 47 – 50, 137, 144, 170, 307; univariate 17; multivariate 140, 220, 228, 309 Norman 182, 211 Norris 5, 6, 8, 14, 24, 28, 36, 44, 45, 49, 77, 107 – 8, 118, 122 – 3, 126, 160, 180 Normed Fit Index (NFI) 228 – 30, 233 Norušis 252, 274 Novomestky 56, 76 Ntzouflas 329, 345 null hypothesis significance testing (NHST) 6, 10, 15, 25 – 31, 36, 106 oblique rotation: direct oblimin 197; promax 197 observed scores 14, 214, 217, 236 observed variables 14, 182, 214 – 15, 217, 221 – 3, 231 Ockey 216, 239, 242 odds ratio 36 Oh 114, 127 Olkin 113, 122, 125 Onsman 209, 212 operationalization: construct, of 215; proficiency, of 329 ordered means 15, 329 ordinal scale 10, 88, 214, 225, 277, 305 354 Index Ortega 5, 8, 24, 36, 45, 49, 77, 107 – 8, 122 – 3, 126, 160, 162, 180 Orthogonal rotation: varimax 197; quartimax 197; equamax 197 Orwin 111, 113, 126 Osborne 194, 197, 210 Oswald 6, 8, 12, 24, 37 – 8, 44, 45, 92, 107, 108, 109, 112 – 14, 115, 117 – 8, 123 – 4, 127, 128 outfit mean square 288 outliers 17 – 18, 48, 51, 94, 115, 137, 140, 220, 288, 309 – 10 Oxford 244, 268, 274 pairwise comparison 49, 67, 69 – 70, 149, 198 Papi 183, 188, 190, 194 – 7, 200 – 1, 203, 205, 209 – 10, 211 parameters: constrained 222; fixed 222; free 222; estimates 222, 224 parametric statistics 49, 162 Parsimony Normed Fit Index (PNFI) 228, 230, 233 partial correlations 13 partial credit model 275, 277 – 8, 284, 292 – 3, 295 – 6, 301, 303 path diagram 79, 228 – 9 Patton 244, 274 Pearson 107, 127 percentage: cumulative 194; descriptive 36, 90, 93, 110, 290, 292, 313, 320, 323; variance, of 194, 197 percentile 37, 51, 90 – 2, 118 person ability 285, 287, 332 Pett 187, 193, 199, 203, 205 – 6, 209, 211 Pexman 82, 105 Phakiti 47 – 8, 77 Pickering 272 – 3 pie chart 79, 97 – 9 Pigott 108, 114, 125, 128 pilot study 290 Pinheiro 175, 181 planned comparisons 330 Plonsky 3, 4, 5, 7, 8, 10, 11, 12, 24, 29, 30, 32, 33, 37 – 8, 42, 44, 45, 46 – 9, 65, 76 – 7, 82, 90, 92, 97, 105, 107, 108 – 9, 110, 112 – 14, 117 – 9, 123 – 4, 127, 127, 128, 133, 158, 160, 162, 181, 182, 187 – 8, 194, 196 – 7, 205, 211, 214, 242, 244, 274, 306, 327 plots 12, 65, 78, 90 – 4, 124, 138, 140, 166 – 7 Poltavtchenko 108, 127 population: covariance 316; distribution 62; effect 114, 135; mean 39; true 26, 33 Porte 5, 8, 45, 77 posterior model probability (PMP) 338, 343 post hoc: comparisons 10 – 11, 330; power 29, 47, 49; tests 48, 334 posttest 39, 110, 112 – 13 power 135 power analysis 29, 43 – 4, 125, 239 Powers 80 – 1, 105 practical significance 27, 38, 43, 107, 114, 117 – 18, 126 precision: measurement, of 10, 90, 115; observation, of 135; statistics, in 30, 74, 133, 137 predictor variables 15, 35; cluster analysis, in 247 – 8, 260 – 1, 266; discriminant analysis, in 307 – 12, 314 – 16, 320 – 1, 323; multiple regression analysis, in 131 – 3, 135 pretest 110, 112 – 13 Principal Axis Factoring (PAF) 184, 193, 208 Principal Components Analysis (PCA) 182, 184 – 5, 187, 193, 294 – 5 prior estimation 341 probability: level 11, 135 – 7, 144, 148, 318; posterior model 338 – 9, 343 Processuse 310, 315 – 20 publication bias 38, 115 – 17, 119 – 24 Purdie 118, 125 p-value 78, 223 – 4, 228, 230, 233 Q-Q plot 61 – 2, 65 – 8, 167 quantiles 51, 65 – 8, 167 Quene 162, 178, 181 questionnaire analysis 294 random assignment 110 random effects: crossed 161, 179, 181; nested 160 – 1, 168 random intercepts 169 – 71, 173, 176 – 7 random slopes 171 – 3, 176 – 8 range restriction 113, 122 Rasch analysis: many-facet model 302; simple model 275 – 8, 296, 301 raters: multiple 279 – 80; severity 278 ratio 94, 97, 188, 223; aspect 102 – 3; F147 – 8; odds 36; scale 10, 14 Raudenbush 160, 161, 162, 181 Raykov 239, 242 R development core team 7 – 8, 164, 181 Index 355 reaction time 170, 220 Reddon 196, 211 regression: coefficients 131, 148 – 9, 153; hierarchical 141; model 33, 131, 147 – 8, 151 – 2, 156, 219; standard 141, 149, 151 – 2; statistical 143; weights 110, 152 – 3 reliability 10, 18, 51, 110, 113 – 14, 135, 153, 221, 290, 299, 309 repeated measures 15, 160, 162, 171, 179 – 80 replication 24, 57, 108, 112, 173, 206, 221, 224 reporting standards 122 resampling 46 – 7, 50 – 1, 72, 75 research synthesis 43, 123 residual: ANOVA, in 71, 148, 153; dimensionality, establishing 294 – 5; linearity, checking for 140, 144; sampling 50 – 1; scaled 169 – 70; square 229, 230, 233; standardized 145 – 6; variance of 170, 217; z 144 Revelle 164, 181 r family of effect sizes 38; see correlation; effect sizes Rietveld 182, 186 – 7, 202, 209, 211 Ripley 56, 76 R-matrix 188 – 9, 192 Robbins 78, 82, 105 Roberts 185, 201, 203, 211 Robson 327 robust statistics 75 Rogers 46 – 7 Root mean square error 223 Root Mean Square Error of Approximation (RMSEA) 223 Rosenkjar 327 Rosenthal 28, 45, 122, 125, 127 Rosnow 28, 45 Ross 5, 8, 107, 127 Rosseel 238, 242 rotation: oblique 197 – 8; orthogonal 197 Rothstein 108, 115, 122, 125, 127, 128 Rovine 219, 242 R programming language R packages: boot 56; discriminant analysis, for 325; ggplot2 101; lattice 101; lavaan 238; lme4 164, 166; moments 56; plyr 56 Rubin 272 – 4 R squared: adjusted 147, 151; change in 132, 149, 151 – 2; see effect sizes Russell 108, 127 Sabatini 158 Sachs 44 Sagan 40, 45 Saito 108, 118, 126 sample: appropriate 157, 187 – 8; independent 74, 113; large 16 – 17, 30, 40, 115, 187, 321, 343; small 11, 16, 29, 32, 46 – 7, 106, 287, 289; suggested 187 – 8 sampling: adequacy 187 – 8, 192; distribution 18, 50, 62, 67; error 29 – 30, 106, 113 – 15; procedures; 50; level of 161 Sarkar 101, 105 Sawaki 158 scale: ability 276; continuous 14; criterion 297; interval 10, 287, 308; Likert 110, 183, 187, 247, 278; level 334; logit 176, 277, 285, 287, 289 – 91, 296, 331; ordinal 10, 88, 214, 225, 277, 305; original 58, 62, 65, 67, 72 – 3; predefined 226; ratio 10; reading 83 – 5; score 84 Scarpello 196, 211 scatterplot 145 – 6 Scheepers 171, 179 Schielzeth 171, 181 Schmidt 113, 114, 122, 125, 127 Schmidtke 183, 188, 190, 194 – 7, 200 – 1, 203, 205, 209 – 10, 211 Schmitt 184, 211, 214, 219, 242 Schoonen 5, 8, 183, 214, 219, 220, 222, 239 Schumacker 279, 304 score: abstract 33; decision point 332; distributions 94 – 5, 236; level 247, 265, 266 – 8, 296, 332; composite 201; missing 198, 220; total 287, 289 scree plot 193 – 7, 205, 208, 2585 Seaman 269, 274 SEM see structural equation modeling Shaul 245, 252, 274 Shintani 107, 126 significant effect 37, 47, 107, 315, 321 similarity and distance measures 250 simple resampling method 50 – 1 SIMPLIS 225 – 6, 228, 238, 241 – 2 Singer 162, 168, 181 Skehan 244, 246, 274 skewness 51, 57, 74, 220 slopes: random 171 – 3, 176 – 8 Snijders 160, 181 356 Index software 6 – 7; AMOS 7, 184, 215, 219, 222, 233 – 5; LISREL 7, 196, 213, 215, 222, 225, 227 – 9; Mplus 7, 236 – 9; PRELIS 225 – 6; SIMPLIS 225 – 6, 228; see also SPSS solutions: 2-cluster 260, 261, 267; 3-cluster 260, 261, 267; 4-cluster 260, 261, 267; six-factor 194, 196, 208; two-factor 196 – 7 Smith, E. V 301 Smith, R. M 301 Sörbom 213 – 14, 225, 228, 238, 242 Spada 108, 118, 127 sparklines 97, 98, 99 Sparks 244, 274 Speigelhaler 329, 341, 345 sphericity 162, 188, 192, 205 Spino 183, 188, 190, 194 – 7, 200 – 1, 203, 205, 209 – 10, 211 SPSS 4, 7, 31, 33, 35, 39, 41, 156, 219; bootstrapping 49, 51 – 2, 54, 57, 60 – 5, 67, 73; cluster analysis 246, 248, 253, 256, 260 – 1; commands 137 – 8, 143, 149; discriminant analysis 307 – 8, 310 – 12, 314 – 15; factor analysis, performing 184, 187, 190, 193, 196 – 9, 202 – 3; mixed effects 164 – 5, 178; output 35, 41, 147, 199, 256, 315; output for ANOVA 148, 153; output for factor analysis 199; output for hierarchical regression 152 – 3; output for regression 147, 149; output for tolerance statistics 140; output for variables 147, 152; Rasch analysis 279; structural equation modeling 219, 225, 233, 238; visual displays 79, 99 – 100, 123 – 4 Squared Euclidean distance 253 standard error 16, 54, 60 – 6, 69, 83, 133 – 4, 170, 220, 228, 231, 234, 287, 296 standardized coefficients 33, 148 statistical power 9 – 11, 23, 29 – 30, 47 – 8, 108, 160, 220; analyzing 29; NHST 30; sample size 30 statistical significance 23 – 30, 36, 43, 49, 72, 92, 117, 135, 143, 147, 151, 152, 173, 174, 223, 323, 329; flaws associated with 24 – 8, 36, 106 – 7; statistical power 30 Steiger 242, 244 stepwise regression 143 Sterling 183, 188, 190, 194 – 7, 200 – 1, 203, 205, 209 – 10, 211 Stevens 27 – 8, 45, 117, 128 Stoel 214, 219, 242 Stoll 245, 274 Strahan 185, 210 Streiner 182, 211 structural equation modeling (SEM) 6, 14, 183, 213; software for study quality 8, 110, 114 Stukas 117, 227 Sullivan 187, 193, 199, 203, 205 – 6, 209, 211 sum of squares 148, 153, 252, 256 – 7 survey questions 183, 203, 268 – 9, 309 – 10 Sutton 115, 122, 128 synthetic ethic: 24, 42 – 3 Tabachnick 131 – 2, 137 – 8, 140 – 1, 143, 157 – 8, 183, 185, 187, 192, 211 table: titles 83; correlations 60, 315; output 63, 275, 279, 284, 301 – 2 Tafaghodtari 207 – 8, 128 Tait 185, 187, 196, 205, 210 Takeuchi 184, 211 Tatham 187, 210 Taylor 27 – 8, 45, 117, 128 test development 222, 234, 237 – 8 Thomas 329, 341, 345 Thompson 183 – 4, 187, 193 – 4, 201, 202 – 3, 211 three-dimensional display 102 Tibshirani 50, 75 Tily 171, 173 – 4, 179 Tobias 109, 113, 126 Tolerance 138 – 40 Tomita 118, 127 transparency 5, 205 treatment 51, 80, 119, 147, 222, 243, 325; coding in R 166, 168 Truscott 108, 128 Tseng 184, 211, 214, 219, 242 Tsuda 244, 274 t-test 11, 19, 25, 27, 49, 51, 160, 162; bootstrapping, in 63, 65 – 6 Tufte 78, 82, 90, 97, 103 Tukey 47, 76 Turner 178, 181 Type I error 4, 11, 49, 77, 314 Type II error 47 – 8, 65 Uchikoshi 245, 252, 274 Ullman 214, 222 – 3, 238 – 9, 242 unstandardized coefficients 35, 148 – 9, 152 – 3 Index 357 Valentine 108, 122, 125, 128 validity 10, 48, 109, 215, 221, 330 – 2 value: absolute 170, 198; exact 83, 86; median 90, 306; observed 131, 135; predicted 71, 131, 145 – 6 van den Burgh 162, 178, 181 Vandergrift 207 – 8, 128 Van Gelderen 214, 219, 242 Van Hout 182, 186 – 7, 202, 209, 211 variable: correlated 188, 203, 310; criterion 6, 33, 131, 154 – 5, 264, 266, 307; individual 220, 324; linguistic 240, 245, 262; moderator 13, 114, 120; redundant 13; standardized 33, 253 – 255; theoretical 215, 217; transforming 18, 253, 255; underlying 203, 217, 221 variance: analysis of 10, 38, 329, 334 – 5, 344; amount of 6, 72, 131, 151, 170, 182, 194, 317; random 159, 161, 168, 171 – 2; shared 35; total 194 – 5, 207 – 8, 257, 316 variance inflation factor (VIF) 138 Velicer 196, 212 Ventura 51, 76 Vergeer 222, 242 von Randow 299, 303 Weisberg 164, 180 Wells 168, 179 West 220, 133 – 6, 156, 158, 242 White 109, 113, 128 Widaman 185, 186, 211, 212 Wikham 105 Wilcox 47, 75 – 7 Wilkinson 78, 80, 86, 105 Wilks’ Lambda 315, 317, 322 Willet 168, 181 Williams 209 Willms 162, 179 Wilson, D. B 31, 110, 110 – 12, 121, 122, 126 Wilson, S 109, 113, 126 Winke 183, 212 Winsteps 7, 278 – 81, 283, 290 – 1, 301 – 4 Wolfe 48, 77 Wood 113 Wright 276 – 7, 278, 287, 304 Wright map 285 – 7, 291, 294, 296, 303 written texts 247 – 8, 325 Ward’s method 252 – 3 Way 269 Weaver 294 – 5, 304 Wegener 185, 192, 209, 210 Zhang 185, 187, 211 ZRESID 144 ZPRED 144 z-score 31, 36, 137, 317 Yamamori 244, 268, 274 Yamashita 113, 125 Yung 46, 76 .. .ADVANCING QUANTITATIVE METHODS IN SECOND LANGUAGE RESEARCH By picking up where introductory texts have left off, Advancing Quantitative ? ?Methods in Second Language Research provides a ? ?second. .. Learning Research (2013) Jegerski/VanPatten Research Methods in Second Language Psycholinguistics (2013) Larson-Hall A Guide to Doing Statistics in Second Language Research Using SPSS and R, Second. .. Plonsky Advancing Quantitative Methods in Second Language Research (2015) Of Related Interest: Gass Input, Interaction, and the Second Language Learner (1997) Gass/Sorace/Selinker Second Language