Applied Communication Research Methods Applied Communication Research Methods: Getting Started as a Researcher demonstrates how to apply concepts to research problems, issues, projects, and questions that communication practitioners face every day Recognizing that students engage more directly with research methods when they experience research through hands-on practice, authors Michael Boyle and Mike Schmierbach developed this text to demonstrate the relevance of research in professional roles and communication careers Along with its distinctive approach to research methods instruction, this text also serves as an enhanced glossary and a superior reference Students can easily navigate key concepts and terminology, which are linked to practical exercises within the context of the instruction In-unit activities and features provide numerous opportunities to delve further into topics covered in class, including: • • • • • Research in Depth—examples of a concept being used in scholarly research Reflect and React—thought-provoking problems and issues that promote reflection and discussion Voices from Industry—Q&As with professionals working in communication industries End-of-unit activities—exercises that reinforce concepts and content Online resources, including sample syllabi, test banks, and more, are available on the companion website:www.routledge.com/cw/boyle Applied Communication Research Methods is a concise, engaging work that today’s students and industry practitioners will embrace and keep on-hand throughout their careers Michael P Boyle is an associate professor at West Chester University and primarily teaches research methods and video production courses Mike Schmierbach is an associate professor at Penn State and teaches courses on political communication, research methods and media effects This page intentionally left blank Applied Communication Research Methods Getting Started as a Researcher Michael P Boyle Mike Schmierbach First published 2015 by Routledge 711 Third Avenue, New York, NY 10017 and by Routledge Park Square, Milton Park, Abingdon, Oxon, OX14 4RN Routledge is an imprint of the Taylor & Francis Group, an informa business © 2015 Taylor & Francis The right of Michael P Boyle and Mike Schmierbach to be identified as authors of this work has been asserted by them in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988 All rights reserved No part of this book may be reprinted or reproduced or utilized in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe Library of Congress Cataloging-in-Publication Data Boyle, Michael P., 1975– Applied communication research methods; everything you need to get started / by Michael P Boyle, Mike Schmierbach pages cm (pbk : alk paper) Communication—Research Communication—Methodology I Schmierbach, Mike, 1976– II Title P91.3.B69 2015 302.2072—dc23 2014026949 ISBN: 978-1-138-85360-7 (hbk) ISBN: 978-0-7656-4233-2 (pbk) ISBN: 978-1-315-71864-4 (ebk) Typeset in Sabon by Apex CoVantage, LLC Unit Overview Preface Basic Principles of Research and a Guide to Using This Book xxiii 2 Basic Concepts of Research 12 Theory and Reading Scholarly Research 46 Ethical Research 80 Concept Explication and Measurement 98 Reliability and Validity 118 Effective Measurement 146 Sampling 176 Experiments and Threats to Validity 210 10 Survey Research 242 11 Content Analysis 266 12 Qualitative Research 284 13 Qualitative Data Analysis 312 14 Descriptive Statistics 332 15 Principles of Inferential Statistics 356 16 Multivariate Inferential Statistics 392 About the Authors 417 Index 419 This page intentionally left blank Contents Preface xxiii Basic Principles of Research and a Guide to Using This Book Principles of Research Empirical Systematic Intersubjective and Replicable Cyclical and Self-Correcting Steps to Success: The Research Process Using This Book Voices from Industry: Jessica and Ziggy Zubric—Customer Experience Consultants 10 Suggested Readings for Further Exploration of Unit Topics Examples 11 Advanced Discussion 11 Basic Concepts of Research Key Terms from Unit Relationship Variable 15 Prediction 16 Unit of Analysis 16 11 12 12 13 C O N T E N T S Types of Variables 17 Dependent Variable 18 Independent Variable 18 Third Variable 19 Ways of Knowing 20 Authority 20 Intuition 21 Research in Depth: The Power of Authority 22 Reflect & React: Counterintuitive Findings 23 Tenacity 23 Goals of Research 24 Application 24 Exploration 25 Description 26 Explanation 26 Classifying Types of Research 27 Steps to Success: Goals of Research 28 Ideographic Versus Nomothetic 29 Qualitative Versus Quantitative 30 A Scientific Approach 31 Voices from Industry: Pamela Denlinger—Director of Account Management at Synapse Marketing Solutions Reflect & React: Using Research to Prepare an Effective Wedding Toast 34 Bias 35 Objective 35 Replication 36 Triangulation 37 Activities 32 39 Activity 2A: Triangulation in Everyday Life 39 Activity 2B: Comparing Qualitative and Quantitative Research 40 Activity 2C: Knowledge Gaining 41 Activity 2D: Evaluating Research Applications 42 Activity 2E: Independent and Dependent Variables 42 References Suggested Readings for Further Exploration of Unit Topics viii Examples 43 Advanced Discussion 43 43 43 C O N T E N T S Theory and Reading Scholarly Research 46 Key Terms from Unit Theory 46 47 Falsification 49 Research in Depth: Theories in Communication Parsimony 51 Scope 52 Research Questions 52 Hypotheses 53 50 Types of Relationships 54 Reflect & React: Developing and Testing Theories Positive Versus Negative Relationship 55 Mediation 56 Moderation 58 Causality and Requirements 58 Reflect & React: Moderation 59 55 Inductive Versus Deductive Reasoning 61 Reflect & React: Assessing Causality 62 Voices from Industry: Chris Nietupski—Communication Consultant 63 Parts of a Research Article 64 Steps to Success: Effectively Reading a Research Article Abstract 66 Literature Review 67 Method Section 67 Results Section 68 Discussion Section 68 Future Directions 69 Reference List 70 Appendix (or Appendices) 70 Tables 71 Figures 72 65 Other Types of Articles Reflect & React: Effective Tables and Figures Literature Review Article 73 Meta-Analysis 74 72 73 Activities Activity 3A: Generating a Topic and Research Question 75 Activity 3B: Literature Search 75 Activity 3C: Identifying and Evaluating the Parts of an Article Activity 3D: Theory Evaluation 76 75 76 ix M U L T I V A R I A T E I N F E R E N T I A L S T A T I S T I C S or “+1” button Ordinary regression would not be appropriate for such a test because the dependent variable is dichotomous But logistic regression could be used You might predict that evaluations of whether the content was enjoyable, credible, and interesting would all uniquely contribute to willingness to click the button By running the analyses, you might find significant positive coefficients for enjoyment and interest but no unique effect of credibility As with any regression analysis, this would show which variables have the clearest, most consistent relationship once you control for the influence of all other variables regression Unit 16 standardized coefficients Unit 16 independent variable Unit logistic regression Unit 16 variable Unit positive versus negative relationship Unit 10 Odds Ratio In ordinary least squares regression, we can evaluate the standardized coefficient to get a better sense of the relative influence of different independent variables by imposing a shared scale on those measures Logistic regression does not produce a standardized coefficient However, the analysis does produce an odds ratio, which can be used to compare the influence of different variables in a similar fashion and offers greater utility First we must understand the idea of odds, which reflect the probability that something will occur divided by the probability it will not For example, a fair coin has odds of that it will come up heads There is a probability it will come up heads and a probability it will come up tails As / = 1, the odds are 1, or perfectly even When the odds are greater than 1, we are saying the event is more likely to occur than not occur For example, the odds of seeing a presidential advertisement during an election are far greater than If 95 percent of voters are exposed to at least one ad, the odds that any given voter will see an ad would be 95 / 05, or 19 On the other hand, when the odds are less than 1, we are saying the event is less likely to occur than not occur For example, the odds that a given individual will attend a political rally during an election are likely pretty small If only percent of people go to a rally, then the odds would be 05 / 95, or about 053 The odds ratio evaluates how these odds change as the value of a given independent variable changes For example, while the average person might have small odds of going to a rally, someone who has volunteered for the campaign might be much more likely to go If nonvolunteers have odds of 05 but volunteers have odds of (still less than even, but considerably higher), the odds ratio would be 10—that is, the individual is 10 times more likely to go if they volunteered than if they didn’t The odds ratio thus represents the amount you have to multiply the first set of odds by to get the second set for each one-unit increase in the IV Here, because the ratio is positive, it means that as the IV (volunteering) increases, the odds of the DV also increase This is a positive relationship On the other hand, consider the example of seeing an advertisement Because ads tend to appear more on traditional media, we might expect that people who spend lots of time on social networking sites would be less likely to see an ad Suppose that for someone M U L T I V A R I A T E I N F E R E N T I A L S T A T I S T I C S who spent hour on social networking sites, the odds of seeing an ad are 19, but for someone who spent hours, they are 9.5 The odds ratio would be 5—if you multiply 19 by 5, you get 9.5 We would further expect that when the hours increased from to 3, the odds would again drop—from 9.5 to 4.75 (9.5 × = 4.75) This would be a negative relationship, as the odds ratio is less than The odds ratio ranges from virtually (for a very negative relationship) to essentially an infinitely high positive number (for a very positive relationship) Steps to Success: Selecting the Appropriate Statistical Test Box — Start here: What level of measurement is your independent variable? Interval or ratio Go to Box Nominal What level of measurement is your dependent variable? Do you have more than one IV, including control variables? Nominal No Interval or ratio Chi-square test Yes Go to Box No Are all your IVs and your DV dichotomous (only two values)? Yes Stop: You cannot analyze these variables Select different measures or recode the variables 11 M U L T I V A R I A T E I N F E R E N T I A L No S T A T I S T I C S Box — Start here: Is/are your IV(s) dichotomous (just two values)? Do you have any IVs that are interval/ratio (including control variables)? Logistic regression Yes Do you have more than one IV (including control variables)? No No ANOVA Yes t-test Yes Regression ANCOVA (interval/ratio variables as covariates) 12 M U L T I V A R I A T E Nominal Box — Start here: What level of measurement is your DV? Is your DV dichotomous (just two values)? Yes I N F E R E N T I A L S T A T I S T I C S Interval/ratio Do you have more than one IV? No No Stop: You cannot analyze these variables Select different measures or recode the variables Correlation (Pearson’s r ) Yes Is your DV nominal? Yes No Regression Logistic regression 13 M U L T I V A R I A T E I N F E R E N T I A L S T A T I S T I C S Activities Activity 16A: Preparing for Multivariate Statistical Analysis See terms: multivariate analysis; inferential statistics; level of measurement For this activity, develop a research question that you would like to answer that considers the relationship among three or more variables Perhaps there is a variable you think is important to consider as a control variable, or perhaps you would like to explore the relationship between multiple independent variables and a single dependent variable As you develop this research question, think about how you will measure the variables and the statistical tests that will be most appropriate for testing your relationship Consider the following questions: ■ ■ ■ ■ Which of the variables will be the dependent variable? Independent? Control? Based on these initial thoughts, what types of statistical analyses would you likely run to explore the relationships among the different variables? What makes this test appropriate? What assumptions does this test have? How would you ensure that your data conform to those assumptions? Activity 16B: Critically Assessing Statistical Analysis See terms: statistical analysis; multivariate analysis; level of measurement For this activity, locate a research article that conducts some type of multivariate analysis Start by describing the nature of the relationship (or relationships) and the test (or tests) that are used to assess the relationship Use the following questions to guide your assessment: ■ ■ ■ ■ ■ ■ 14 What particular tests are used in this research report? Identify each of the different types of variables (i.e., independent, dependent, and control) What is the level of measurement for each of those variables? Are these particular tests appropriate? Why or why not? Could the researchers have used different tests to test these relationships? How would you evaluate the quality as far as how the tests and their results are reported? Do the researchers clearly describe the results of their tests? Is there any information missing that would help your evaluation of the data and the results as described? Are the conclusions drawn from the data accurate based on the tests conducted? M U L T I V A R I A T E I N F E R E N T I A L S T A T I S T I C S Activity 16C: Determining Appropriate Statistical Tests See terms: chi-square; t-test; regression; correlation; ANOVA For the following research scenarios, determine the appropriate statistical test you would use to test the relationships described In some cases, there may be multiple options that are appropriate Therefore, be sure to indicate a clear rationale why you have selected the test you did Finally, consider the assumptions of each test in making your decisions Research scenario Appropriate test You are interested in comparing test scores for the final exam in a research methods course for three different groups Group receives a weekly personal tutoring session beyond the regular course content Group receives enhanced course materials ahead of lecture Group receives both the weekly personal tutoring and the enhanced course materials You work for a political campaign and are interested in testing the relationship between age and the amount of money donated to the candidate for whom you work As part of your work for an advertising agency, you have been asked to explore the relationship between gender and media preferences In particular, the agency would like to know whether there are significant differences between men and women in their favorite social media platform As part of your work for a nonprofit agency, you are interested in understanding whether the likeability of the agency’s spokesperson is related to the amount of money donated by individuals However, you expect that individual attitudes toward philanthropy and community volunteering also affect the amount of money donated Your director has asked you to control for those two variables in assessing the relationship between spokesperson likeability and money donated 15 M U L T I V A R I A T E I N F E R E N T I A L S T A T I S T I C S Suggested Readings for Further Exploration of Unit 16 Topics Examples Eveland, W.P., Jr., Hayes, A.F., Shah, D.V., & Kwak, N (2005) Understanding the relationship between communication and political knowledge: A model comparison approach using panel data Political Communication, 22(4), 423–446 doi:10.1080/10584600500311345 Kahlor, L., & Eastin, M.S (2011) Television’s role in the culture of violence toward women: A study of television viewing and the cultivation of rape myth acceptance in the United States Journal of Broadcasting & Electronic Media, 55(2), 215–231 doi:10.1080/0883 8151.2011.566085 Advanced Discussion Hayes, A.F (2009) Beyond Baron and Kenny: Statistical mediation analysis in the new millennium Communication Monographs, 76(4), 408–420 doi:10.1080/03637750903310360 Hayes, A.F., Glynn, C.J., and & Huge, M.E (2012) Cautions regarding the interpretation of regression coefficients and hypothesis tests in linear models with interactions Communication Methods and Measures, 6(1), 1–11 doi:10.1080/19312458.2012.651415 Hayes, A.F., & Matthes, J (2009) Computational procedures for probing interactions in OLS and logistic regression: SPSS and SAS implementations Behavior Research Methods, 41(3), 924–936 doi:10.3758/BRM.41.3.924 Mundry, R., & Nunn, C.L (2009) Stepwise model fitting and statistical inference: Turning noise into signal pollution The American Naturalist, 173(1), 119–123 doi:10.1086/593303 Preacher, K.J., & Hayes, A.F (2008) Assessing mediation in communication research In A.F Hayes, M.D Slater, & L.B Snyder (Eds.), The Sage sourcebook of advanced data analysis methods for communication research (pp 13–54) Thousand Oaks, CA: Sage Publications 16 About the Authors Michael P Boyle (PhD, University of Wisconsin–Madison) primarily teaches research methods and video production courses His research focuses on how the news media presents issues to the public, how and why people use media, and the implications of news coverage and media use patterns on outcomes such as information seeking, public opinion, and perceptions of media effects He is a happily married father of two who enjoys spending time with his family, playing guitar, and drinking good beer Mike Schmierbach (PhD, University of Wisconsin–Madison) teaches courses on political communication, research methods and media effects His research focuses on the factors that contribute to perceptions of media, including beliefs about media influence as well as enjoyment Much of this work considers video games and other interactive technologies He is a proud “father” of several pets and enjoys hiking and cooking in his spare time This page intentionally left blank Index Note: Bolded page numbers refer to the main entry for a listed term Entries in italics indicate Figures abstract 66 anchor 110, 160, 162, 171 ANCOVA (analysis of covariance) 398, 400; see also ANOVA anonymity 87, 88, 249, 255; see also privacy ANOVA 371, 380–81, 398–9, 400–02; one-way 378–80 appendix 68, 70, 72 application 10, 24–5, 26, 28, 58–9, 87 artifact see social artifact attribute 20, 106, 221, 319, 374, 378 authority 5–6, 20–1, 22, 23 bell curve see normal distribution between-subjects design 214–15, 216, 218–19 bias 4, 21, 35, 122, 168, 170, 194–5, 201; interparticipant 222, 229–30; publication 74; researcher 87, 154, 169, 222, 233–4, 274, 286; selection 215, 234–35; see also validity bivariate analysis 373, 393–6; see also ANOVA; Chi-square; correlation; t-test Bonferroni correction 367–8, 402; see also Type I error case study 287–8, 288 causality 18–19, 26–7, 28, 58–61, 62, 135, 213, 217, 395; time order 60, 213, 217, 245, 257–9; see also control variable; relationship; spurious ceiling effect 232–3; see also validity cell-phone-only household 184, 250–1, 252–3 census 186–7, 192 central tendency 334–5, 337, 345–7; mean 335–6, 340– 1, 342–3, 345–7, 348–9, 349, 369, 371–2, 376–7, 379, 407; median 336–7, 339–340, 345–7, 348–9, 349; mode 337–8, 345; Chi-square 364, 373, 374–6, 376 clarity 136, 168–9, 170 closed-ended 155–6, 163–4, 165–6; see also open-ended cluster sampling 185, 202, 203; see also probability sampling coding 130–1, 163–5, 222, 269, 272–3, 274, 276, 278–9, 292, 316–8, 324; guide 269–70, 277, 319; sheet 270 coefficient: standardized 407–8, 410; unstandardized 403, 406–7, 409 coerce 84, 181, 182 cohort 258; see also longitudinal design comparison group see treatment group complete observer see participant observation complete participant see participant observation completion rate 10, 181, 249, 251, 255 concept 30, 67, 100–1, 102, 103, 105, 124–5, 127, 128, 130–1, 132–3, 136–7, 137–8, 163–4, 169, 269, 302, 313, 317, 319, 401; see also concept explication concept explication 67, 99–100, 103, 104, 120, 128, 130, 134, 136 conceptual definition 101, 102, 103, 119, 130–1, 134, 169, 269, 273; see also operational definition confederate 88–9, 90, 219–20 I N D E X confidence interval 180, 182, 188–9, 190, 193–4, 202, 336, 341, 346, 357; see also estimation; sampling distribution confidence level 188, 190, 192, 193, 346; see also estimation; sampling distribution confidentiality 87–88, 249; see also privacy consent see informed consent consent form 83, 85, 86, 157 constant comparative technique 318–20; see also grounded theory construct validity see validity content analysis 150, 185, 187, 205, 267–8, 319, 323 content validity see validity contingency question 160–1, 244, 253; see also filter question control group 222–3, 227, 230, 235, 378 convenience sampling 195–6, 197, 198; see also nonprobability sampling conversation analysis 314–16, 325 correlation (Pearson's r) 363–4, 366, 369, 372, 381–4, 383, 408; see also bivariate analysis criterion validity see validity critical value see significance Cronbach's Alpha 124, 137, 138–40, 140–1 cross-sectional design 245–6; see also longitudinal design cyclical 5–6, 24, 31, 92, 363 debriefing 82, 84, 88–9, 90, 157, 291 deception 82, 85, 88, 89–90, 92, 222, 229, 235 deductive reasoning 48, 55, 61–62, 317; see also inductive reasoning deliberate sampling 178, 194, 196, 197, 198, 199, 287 demand characteristics 134, 222, 227, 235, degrees of freedom 68, 358–9, 364, 368–9, 370, 374, 376, 377, 378, 380, 381 dependent variable see variable depth interviews 87, 105, 150, 268, 289, 293, 295, 302–3, 324–5 description 24, 26, 28, 30 descriptive statistics see statistics dimension 100, 102–4, 105–6, 124, 130, 169, 171, 319–20, 325 direct observation see observation discussion section 54, 66, 68–9 dispersion 338–9, 361; range 20, 67, 107, 110, 171, 188, 193, 339–40, 342, 344, 407; standard deviation 67, 111, 193, 340–1, 342–3, 346–7, 377, 407; variance 107, 189, 340, 378–9, 380, 394, 398, 405; see also variance explained double-barreled question 160, 169–70 double-blind design 222, 228, 233–4 ecological validity see external validity effect size 68, 183–4, 236, 365, 369, 371, 372, 398, 402, 408–9; see also variance explained 420 empirical 3–4, 20, 25, 31, 47–8, 61–2, 104 estimation 187, 190–1, 191, 334 ethnography 286, 289–90; see also field research exemplars 316–7, 325, 326 exhaustive 106, 109, 163–4, 165–6, exiting 290–1, 294 experiments 37, 59–60, 62, 74, 88, 135, 179, 211–12, 216–18, 225, 367, 373, 376, 378 expertise see authority explanation 25, 26–7, 28, 29, 51 exploration 24, 25, 26, 28, 31, 164, 184, 304 external validity 135–6, 194, 216–17, 219, 227–8, 231, 243; ecological validity 227; see also validity F-statistic 373, 378–9, 380–1, 398, 400 face validity see validity face-to-face survey 202, 249–50; see also interview-style survey factor analysis 104, 130, 137 factorial design 219–21, 221 falsification 49–51, 54, 92 field experiment 217–8, 289 field notes 289, 291–2, 294, 317, 326 field research 288–9, 291, 293–4, 299 figures 66, 71, 72, 73 filter question 157, 158, 253, 255; see also contingency question floor effect 232–3; see also validity focus group 34, 37, 150, 181, 233, 294, 295–6, 297, 298, 316, 325, 327; see also moderator frequency distribution 342–3, 347–8; see also histogram fully structured interview see interview structure funnel 302, 303 future directions 66, 69; see also discussion section generalizability see external validity grounded theory 317–18, 320, 324 group equivalence see random assignment harm 82–3, 85, 86, 88–90 Hawethorne effect 227–8; see also validity hierarchical regression see regression histogram 343–5 historical analysis 277, 320–1 history 228–9; see also validity honesty 91–2, 93 hypotheses 50–1, 53–4, 55, 61–2, 65–6, 67–8, 92, 137, 152, 361–3, 369–70, 376–7 ideographic 29–30, 302; see also nomothetic incentive 67, 84, 87, 181–2, 182, 254, 256, 259, 295 independent variable see variable index 124, 137–8, 401; see also Cronbach's Alpha indicator 67–8, 102–104, 103, 105–6, 124–5, 130, 138, 169, 171 I N D E X indirect observation see observation inductive reasoning 48, 55, 61–2, 317, 319; see also deductive reasoning inference 199, 361–2 inferential statistics see statistics informed consent 83–4, 88, 90, 92, 155, 157, 291 institutional review 83, 85, 90–1, 181 interaction 72, 397–8, 399–401; see also moderation intercept see regression intercoder reliability 270, 272–3, 275, 277–8 intercoder statistics 273–5 internal validity 134–5, 194, 225–7, 236; see also validity internet panels 245, 255–6 internet survey 254–5; see also self-administered survey interparticipant bias see bias intersubjective 4–5, 31, 35–6, 101–2, 277; see also replication interval see levels of measurement interview structure 304, 306; fully structured 304; semistructured 304–5; unstructured 305–6 interview-style survey 246–9, 253 intuition 21–2, 23, 23, 149 item-total reliability see reliability journal article 47, 64–5, 65–6, 70, 225 Krippendorff's Alpha see intercoder statistics kurtosis 343, 347, 347 latent content 276, 277–8, 278, 323;see also manifest content leading question 160, 168 levels of measurement 106, 107, 108, 273, 335, 337, 373, 411–3; interval 106, 110–11, 171, 335, 381, 398, 402; nominal 106, 109, 164, 337, 373–4, 376, 378, 398, 400–1; ordinal 109–10, 336–7, 339, 345; ratio 111, 335, 339, 345, 373, 376, 378, 381, 398, 400–2 likert-type item 171, 406 linearity 382, 384–5, 384–5, 402, 406 literature review 65, 67, 69, 93 literature review article 73 logistic regression 409–10 longitudinal design 244, 246, 257–8 mail survey 180, 256–7; see also self-administered survey manifest content 278, 278–9, 323; see also latent content MANOVA 401–2; see also ANOVA margin of error 189–90, 190–1 matched assignment 213–14, 216 maturation 230; see also validity mean see central tendency measurement error 119–21, 123, 147; random 74, 107, 121, 125, 132, 134, 138, 139–40, 180, 193, 215, 224, 363; systematic 122, 128, 134–5, 139, 180, 187, 194, 205, 363 median see central tendency mediation 56–7, 57, 393 meta-analysis 74, 372 method section 65, 67–8 Milgram, S 22, 88–90 mode see central tendency moderation 58, 59, 74, 233; see also interaction moderator 298–9, 327; see also focus group mortality 230–1, 256, 259; see also validity multivariate analysis 357, 393–4, 397; see also ANOVA; ANCOVA, MANOVA, regression mutually exclusive 106, 109, 160, 163, 166–7 natural settings 135, 293–4 negative relationship see relationship network sampling see snowball sampling nominal see levels of measurement nomothetic 29–30 nonprobability sampling 179, 187, 194–5, 198, 199 nonreactive measures see reactives measures normal distribution 338, 341, 343, 345–7, 346, 364, 404 null hypothesis 54, 361, 362–3, 364–7, 369–70, 374, 377, 379, 381, 398 objective 35–6, 99, 274, 279; see also intersubjective observation 3, 48, 84, 149, 151–4, 154, 155, 222, 248, 288, 291–4; direct 149–50; indirect 149–50; see also participant observation observer effect see Hawthorne effect observer-as-participant see participant observation obtrusive measures 149, 150, 151, 268 odds ratio 410–11; see also logistic regression one-tailed test 369–70, 377; see also two-tailed test open-ended 30, 155, 156, 164–5, 166, 249, 267, 270, 276; see also closed-ended operational definition 103, 104–5, 134, 138, 269, 273; see also conceptual definition order effects 157, 162–3 ordinal see levels of measurement others' reports 152 outliers 335–6, 340, 343, 348–9, 348 p-value 71, 361–3, 363–5, 367–70, 374, 376, 377, 380–1, 398–9, 402; see also statistical significance paired assignment see matched assignment paired t-test see t-test panel 60, 62, 251, 259, 260, 321; see also longitudinal design parallel-forms reliability see reliability parameter 179, 186, 188–9, 191–3, 346–7, 357, 361 421 I N D E X parsimony 51 participant observation 84, 292, 299; complete observer 292, 300–1; complete participant 291–2, 301; observer–as-participant 301–2; participant as observer 301–2 participant-as-observer see participant observation peer review 64–5; see also intersubjective percent agreement see intercoder statistics periodicity 204–5 plagiarism 92–3 planned comparison see post hoc test polls 24, 31, 132, 168, 184, 187–90, 191, 198, 244–5, 246, 252–3, 358; tracking poll 245, 250 population 31, 74, 83, 86, 128, 135, 179, 184, 185–6, 187–9, 193, 195–7, 199–200, 204, 227, 231, 243, 245, 258, 267, 334, 342, 345–7, 361–2, 366, 371, 373 positive relationship see relationship post hoc test 184, 378, 380, 399, 401 post-test see pre-test pre-test 223–4, 224–5, 235 prediction see relationship privacy 86, 87, 91, 249–50, 326 probability sampling 187, 194, 199, 204, 244 probing 156, 248–50, 298, 303 publication bias see bias purposive sampling see deliberate sampling qualitative measurement 30–1, 86, 88, 107, 109, 164, 196, 251, 286, 291–2, 313–4, 323 qualitative interviews see depth interviews quantitative measurement 30–31, 68, 86, 88, 105, 109, 124, 163–4, 267, 269, 276, 286–7, 305, 313, 316, 333 quasi-experimental design 213–14, 216–17, 218 questionnaire 70, 87, 147, 155–7, 158–9, 160, 162–3, 165, 170, 246, 249–50, 253–7, 292 quota sampling 196–7, 255; see also nonprobability sampling R2 402 406, 408–9 random 200, 201, 218 random assignment 60, 213, 214, 216–17, 224, 229, 234, 236 random sample see simple random sampling random measurement error see measurement error random number table 184, 200–1, 205 range see dispersion rapport 90, 155–7, 161, 244, 248–50, 251, 253, 256, 286, 291, 302, 305 ratio see levels of measurement reactive measures 150, 153, 155, 289, 300–1 reference list 67, 70 regression 384, 402–4, 404–6, 408–9; hierarchical 408–9; see also coefficient 422 regression toward the mean 231–2, ; see also validity relationship 13–5, 16, 17–9, 25–7, 53–5, 56–9, 74, 104, 121, 125, 128, 132, 134–5, 182, 183–4, 194, 213, 227, 243, 294, 334, 361–3, 367–9, 373, 384, 393–4, 397; negative 55–6, 129–30, 139, 406–7, 410–1; positive 55–6, 129–30, 404, 406–7, 410–1 reliability 67, 121, 123–3, 126, 128, 138, 305; itemtotal 124–5, 138–40; parallel-forms 125–6, 132; splithalves 126–7; test-retest 127 replicable 3, 4–5, 31; see also intersubjective replication 4–5, 15, 34, 36–7, 37, 47, 102, 212, 225–6 representative sample 152, 177–9, 179–80, 182, 185, 187, 194–6, 198, 201–2, 203, 236, 253, 255–6 researcher bias see bias response rate 179, 180–1, 182, 187, 193, 197, 248, 251, 254–5, 256, 257, 259, response set 160, 161–2, 244 results section 53, 64, 65–6, 68, 73, 316 reversed items 139, 160, 161 rhetorical criticism 322–3, 324 sample 36–7, 47, 65, 67, 69, 121, 128, 135, 152, 177–9, 180, 182, 185, 186–7, 189–90, 192–3, 244–5, 250, 254–5, 258–9, 267, 333–4, 346–7, 361–2, 368–9, 378, 407 sample size 65, 180–1, 182, 183–4, 188–9, 190–1, 201–2, 204–5, 245, 255, 259, 269, 362–3, 368–9, 377, 378, 380, 381 sampling distribution 187, 191–3, 193, 194, 201, 346, 347, 357–8, 361–2, 363–4, 368, 369 sampling error 179, 182, 185–6, 189–90, 191, 192, 193, 199, 202, 203, 252, 371 sampling frame180, 184–5, 185, 196, 200–3, 203, 204–5, 250, 255–6 sampling interval 204–5, 205, ; see also systematic sampling saturation 290–1, 294–5, 317–8 scope 51, 52, 61, 69, 211–2 Scott's pi see intercoder statistics secondary analysis 86, 152–3, 276–7 selection bias see bias self-administered survey 147–8, 156, 161, 246–9, 253–4, 254–7 self-correcting see cyclical self-report 121, 149–50, 151–2, 153 semantic differential item 170, 171 semistructured interview see interview structure sensitization 223–5, 235; see also validity significance: practical 365, 368–9, 371–2; statistical 58, 74, 91–2, 184, 361–2, 364, 365–6, 367–8, 377, 379–80, 382, 396–7, 401–3 simple random sampling 179, 184, 185, 187, 192, 199, 201–2; see also probability sampling skew 188–9, 335–7, 342–4, 345–6, 349–50, 350–1, 363–5 I N D E X snowball sampling 194–5, 197–8; see also nonprobability sampling social artifacts 108, 148, 150, 268–9, 276, 276–7, 322–4 social desirability 122–3, 138–40, 151–2, 153–4, 246–9 Solomon four-group design 223–4, 224–5, 236, split-halves reliability see reliability spuriousness 58–61, 62, 395–6, 397–8, 400; see also causality standard deviation see dispersion standard error 187, 189–90, 193–4, 341, 347, 403–4, 406, 409 standardized coefficient see coefficient statistical control 216–7, 357, 394–5 statistical power 183–4, 212, 245, 359, 363, 368 statistics 30, 71, 74, 107, 179, 183–4, 186–7, 191–2, 333–4, 336, 361; descriptive 67 334, ; inferential 54, 68, 213, 357–9, 367 stimulus group see treatment group stratified sampling x; see also probability sampling suppression 396–7 survey research 105, 155, 179, 181, 243–4, 252–3, 286 systematic 3, 4, 14–15, 20, 31, 65–6, 153–4, 199, 267–8, 316 systematic measurement error see measurement error t-tests 341, 367, 369–70, 373, 376–8, 379, 402, 405, 412; see also bivariate analysis tables 65–6, 71, 73, 193 telephone survey 10, 127, 250; see also interview-style survey tenacity 20, 21, 23, 149 test-retest reliability see reliability testing effect 211–2, 235; see also validity textual analysis 322, 323–4 text see social artifacts themes 270, 294–5, 314, 316, 318, 319, 321, 323, 324–5, 326 theory 5, 7, 13–15, 18, 24, 28, 36, 47–9, 50, 52, 52–4, 55, 56, 58, 61–2, 64–5, 73, 100–1, 102, 140–1, 152, 316–19, 324–5, 358, 379–80, 395–6, 401–2 third variable see variable threats to validity 227, 236 time order see causality tracking poll see polls transcript 268, 302–3, 313–15, 317, 318–19, 323, 325–7, 326 treatment group 72, 213, 214–15, 216–17, 218–19, 222–3, 228, 230–2, 373, 376–8 trend 152–3, 257, 258–9; see also longitudinal design triangulation 29, 34, 37–8, 102, 135, 288–9, 298, 299–300 Tuskegee 82–3 two-tailed test 369–70, 377; see also one-tailed test type I error 362, 365–6, 366–7, 368, 379–80, 399, 401–2, ; see also Bonferroni correction; statistical significance type II error 183–4, 367, 368–9, ; see also statistical significance unit of analysis 16–7, 34, 106, 108, 147, 150, 152, 202, 269, 270, 273, 319, 322, 382 unit of observation 17; see also unit of analysis unobtrusive measures see obtrusive measures unstandardized coefficient see coefficient unstructured interview see interview structure validity 84, 106, 122–3, 123, 128, 136–7, 161–2, 254, 305; concurrent 131; construct 128–30; content 105, 130–1; convergent 128–30; criterion 131–2; divergent 130; face 105, 132–4, 141; predictive 131–2 variable 15–16, 16–17, 25–6, 28, 29–30, 34, 52–3, 65, 67, 101, 101, 105–7, 108, 119–20, 124, 128–134, 138, 269–270, 334, 338, 340, 344, 373, 407; control 19–20, 57, 123, 135, 394, 404, 409; dependent 17, 55, 56, 58–60, 62, 214, 216, 223, 257, 367, 371, 382, 394, 397, 401, 402, 404, 408, 409–10; independent 18–19, 55, 56, 58–60, 62, 218–9, 221, 236, 257, 371, 382, 394, 397, 398, 402, 404, 408, 409–10; third 19–20, 56, 58–60, 61, 62, 224, 395–7 variance see dispersion variance explained 371–2, 372, 379, 394–5, 406, 408–9 voluntary participation 84–5, 256, 291 volunteer sampling 194–5, 198–9 ways of knowing 20, 149; see also authority; empirical; intuition; tenacity within-subjects design 214, 216, 224 zero-order see bivariate analysis 423 .. .Applied Communication Research Methods Applied Communication Research Methods: Getting Started as a Researcher demonstrates how to apply concepts to research problems, issues, projects, and... courses on political communication, research methods and media effects This page intentionally left blank Applied Communication Research Methods Getting Started as a? ?Researcher Michael P Boyle Mike. .. Cataloging-in-Publication Data Boyle, Michael P. , 1975– Applied communication research methods; everything you need to get started / by Michael P Boyle, Mike Schmierbach pages cm (pbk : alk paper) Communication? ??Research