ebook THE GUILFORD PRESS Selecting the Right Analyses for Your Data Also Available When to Use What Research Design W Paul Vogt, Dianne C Gardner, and Lynne M. Haeffele Selecting the Right Analyses for Your Data Quantitative, Qualitative, and Mixed Methods W Paul Vogt Elaine R Vogt Dianne C Gardner Lynne M Haeffele THE GUILFORD PRESS New York London © 2014 The Guilford Press A Division of Guilford Publications, Inc 72 Spring Street, New York, NY 10012 www.guilford.com All rights reserved No part of this book may be reproduced, translated, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, microfilming, recording, or otherwise, without written permission from the publisher Printed in the United States of America This book is printed on acid-free paper Last digit is print number: 9 8 7 6 5 4 3 2 1 Library of Congress Cataloging-in-Publication Data is available from the publisher ISBN: 978-1-4625-1576-9 (paperback) ISBN: 978-1-4625-1602-5 (hardcover) Preface and Acknowledgments Using the right analysis methods leads to more justifiable conclusions and more persuasive interpretations of your data Several plausible coding and analysis options exist for any set of data—qualitative, quantitative, or graphic/visual Helping readers select among those options is our goal in this book Because the range of choices is broad, so too is the range of topics we have addressed In addition to the standard division between quantitative and qualitative coding methods and analyses, discussed in specific chapters and sections, we have dealt with graphic data and analyses throughout the book We have also addressed in virtually every chapter the issues involved in combining qualitative, quantitative, and graphic data and techniques in mixed methods approaches We intentionally cover a very large number of topics and consider this a strength of the book; it enables readers to consider a broad range of options in one place Analysis choices are usually tied to prior design and sampling decisions This means that Selecting the Right Analyses for Your Data is naturally tied to topics addressed in our companion volume, When to Use What Research Design, published in 2012 In that book we introduced guidelines for starting along the intricate paths of choices researchers face as they wend their way through a research project Completing the steps of a research project—from the initial idea through formulating a research question, choosing methods of data collection, and identifying populations and sampling methods to deciding how to code, analyze, and interpret the data thus collected—is an arduous process, but few jobs are as rewarding We think of the topic—from the research question to the interpretation of evidence—as a unified whole We have dealt with it in two books, rather than in one huge volume, mostly for logistical reasons The two books are free standing As in a good marriage, they are distinct but happier as a pair It has been exciting to bring to fruition the two-volume project, and we hope that you too will find it useful and occasionally provocative as you select effective methods to collect, code, analyze, and interpret your data v vi Preface and Acknowledgments To assist you with the selection process, the book uses several organizing techniques to help orient readers, which are often called pedagogical features: • Opening chapter previews provide readers with a quick way to find the useful (and often unexpected) topic nuggets in each chapter • End-of-chapter Summary Tables recap the dos and don’ts and the advantages and disadvantages of the various analytic techniques • End-of-chapter Suggestions for Further Reading are provided that include detailed summaries of what readers can find in each one and why they might want to read them for greater depth or more technical information • Chapter 14 concludes the book with aphorisms containing advice on different themes It is a great pleasure to acknowledge the help we have received along the way This book would not have been written without the constant support and advice—from the early planning to the final copyediting—of C Deborah Laughton, Publisher, Methodology and Statistics, at The Guilford Press She also recruited a wonderful group of external reviewers for the manuscript Their suggestions for improving the book were exceptionally helpful These external reviewers were initially anonymous, of course, but now we can thank at least some of them by name: Theresa E DiDonato, Department of Psychology, Loyola University, Baltimore, Maryland; Marji Erickson Warfield, The Heller School for Social Policy and Management, Brandeis University, Waltham, Massachusetts; Janet Salmons, Department of Business, School of Business and Technology, Capella University, Minneapolis, Minnesota; Ryan Spohn, School of Criminology and Criminal Justice, University of Nebraska at Omaha, Omaha, Nebraska; Jerrell C Cassady, Department of Educational Psychology, Ball State University, Muncie, I ndiana; and Tracey LaPierre, Department of Sociology, University of Kansas, Lawrence, Kansas The editorial and production staff at The Guilford Press, especially Anna Nelson, have been wonderful to work with They have been efficient, professional, and friendly as they turned our rough typescript into a polished work This book and its companion volume, When to Use What Research Design, were written with colleagues and students in mind These groups helped in ways too numerous to recount, both directly and indirectly Many of the chapters were field tested in classes on research design and in several courses on data analysis for graduate students at Illinois State University We are especially grateful to students with whom we worked on dissertation committees as well as in classes They inspired us to write in ways that are directly useful for the practice of research We have also had opportunities to learn about research practice from working on several sponsored research projects funded by the U.S Department of Education, the National Science Foundation, and the Lumina Foundation Also important has been the extensive program evaluation work we have done under the auspices of the Illinois Board of Higher Education (mostly funded by the U.S Department of Education) Although we had help from these sources, it remains true, of course, that we alone are responsible for the book’s shortcomings Abbreviations Used in This Book The following is a list of abbreviations used in this book If a term and its abbreviation are used only once, they are defined where they are used American Community Survey AIK Akaike information criterion ANCOVA analysis of covariance ANOVA analysis of variance AUC area under the curve BMI body mass index CAQDAS computer-assisted qualitative data analysis software CART classification and regression trees CDC Centers for Disease Control and Prevention CFA confirmatory (or common) factor analysis CI confidence interval COMPASSS comparative methods for systematic cross-case analysis CPS Current Population Survey CRA correlation and regression analysis CSND cumulative standard normal distribution DA discriminant analysis d-i-d difference-in-difference DIF differential item functioning DOI digital object identifier vii Abbreviations ACS Abbreviations DV dependent variable E estimate or error or error terms EDA exploratory data analysis EFA exploratory factor analysis ELL English language learner ES effect size ESCI effect-size confidence interval FA factor analysis GDP gross domestic product GIS geographic information systems GLM general (and generalized) linear model GPA grade point average GRE Graduate Record Examination GSS general social survey GT grounded theory HLM hierarchical linear modeling HSD honestly significant difference ICC intraclass correlation ICPSR Inter-University Consortium for Political and Social Research IPEDS integrated postsecondary education data system IQ intelligence quotient IQR interquartile range IRB institutional review board IRT item response theory I-T information-theoretic analysis IV independent variable IVE instrumental variable estimation JOB Job Outreach Bureau LGCM latent growth curve modeling LOVE left-out variable error LR logit (or logistic) regression LS least squares viii ...ebook THE GUILFORD PRESS Selecting the Right Analyses for Your Data Also Available When to Use What Research Design W Paul Vogt, Dianne C Gardner, and? ?Lynne M.? ?Haeffele Selecting the Right Analyses. .. Right Analyses for? ?Your Data Quantitative, Qualitative, and? ?Mixed? ?Methods W Paul Vogt Elaine R Vogt Dianne C Gardner Lynne M Haeffele THE GUILFORD PRESS New York London © 2014 The Guilford Press... adequate, and so on These ranks might then be converted into A, B, C, and so forth, and they, in turn, might be converted into numbers 4, 3, 2, and so forth If you sprain your ankle, the sprain