www.it-ebooks.info Dillman www.it-ebooks.info ffirs1.tex V3 - 06/30/2014 9:42 A.M Page ii Dillman ffirs1.tex V3 - 06/30/2014 FOURTH EDITION INTERNET, PHONE, MAIL, AND MIXED-MODE SURVEYS www.it-ebooks.info 9:42 A.M Page i Dillman www.it-ebooks.info ffirs1.tex V3 - 06/30/2014 9:42 A.M Page ii Dillman ffirs1.tex V3 - 06/30/2014 FOURTH EDITION INTERNET, PHONE, MAIL, AND MIXED-MODE SURVEYS The Tailored Design Method Don A Dillman Jolene D Smyth Leah Melani Christian www.it-ebooks.info 9:42 A.M Page iii Dillman ffirs1.tex V3 - 06/30/2014 Cover design: Wiley Cover image: © iStockphoto/khalus Copyright © 2014 by John Wiley & Sons, Inc All rights reserved Published by John Wiley & Sons, Inc., Hoboken, New Jersey Published simultaneously in Canada No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600, or on the web at www.copyright.com Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008 Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose No warranty may be created or extended by sales representatives or written sales materials The advice and strategies contained herein may not be suitable for your situation You should consult with a professional where appropriate Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages This publication is designed to provide accurate and authoritative information in regard to the subject matter covered It is sold with the understanding that the publisher is not engaged in rendering professional services If legal, accounting, medical, psychological or any other expert assistance is required, the services of a competent professional person should be sought Designations used by companies to distinguish their products are often claimed as trademarks In all instances where John Wiley & Sons, Inc is aware of a claim, the product names appear in initial capital or all capital letters Readers, however, should contact the appropriate companies for more complete information regarding trademarks and registration For general information on our other products and services please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002 Wiley publishes in a variety of print and electronic formats and by print-on-demand Some material included with standard print versions of this book may not be included in e-books or in print-on-demand If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download this material at http://booksupport.wiley.com For more information about Wiley products, visit www.wiley.com ISBN: 978-1-118-45614-9 (cloth) ISBN: 978-1-118-92129-6 (ebk) ISBN: 978-1-118-92130-2 (ebk) Printed in the United States of America 10 www.it-ebooks.info 9:42 A.M Page iv Dillman ffirs2.tex V3 - 06/30/2014 To John Tarnai (1947–2012) For his leadership of the Social and Economic Sciences Research Center at Washington State University, the laboratory for our collaborative efforts to develop and test the methods described in this book Janet Harkness (1948–2012) For encouraging the further development of these methods as Director of the Survey Research and Methodology (SRAM) Program at the University of Nebraska–Lincoln www.it-ebooks.info 7:24 A.M Page v Dillman www.it-ebooks.info ffirs2.tex V3 - 06/30/2014 7:24 A.M Page vi Dillman ftoc.tex V3 - 06/30/2014 10:54 A.M Page vii Contents Additional Resources xi Preface xiii Chapter Sample Surveys in Our Electronic World Four Cornerstones of Quality Surveys What Is Different About Surveying in the 2010s? Why Emphasize Mixed-Mode Data Collection? What Is Tailored Design and Why Is It Needed? Conclusion 10 12 15 17 Chapter Reducing People’s Reluctance to Respond to Surveys 19 Example of a Survey With a High Response Rate Using Social Exchange Concepts to Motivate Potential Respondents Putting the Parts Together: Some Guidelines for Applying Social Exchange Mixed-Mode Designs Provide New Opportunities for Applying Social Exchange Returning to the WSU Doctoral Student Experience Survey: Why It Obtained Such a High Response Rate Conclusion List of Guidelines 21 Chapter Covering the Population and Selecting Who to Survey Essential Definitions and Their Use Current Coverage and Access Considerations Common Sampling Frames and Assessing How Well They Cover the Population Probability Sampling Postsurvey Adjustments and Calculating Sampling Error Nonprobability Sampling Conclusion 23 42 47 50 55 55 56 57 58 62 75 87 91 92 vii www.it-ebooks.info Dillman viii ftoc.tex V3 - 06/30/2014 10:54 A.M Page viii Contents Chapter The Fundamentals of Writing Questions 94 Issues to Consider When Starting to Craft Survey Questions The Anatomy of a Survey Question and Types of Question Formats Guidelines for Choosing Words and Forming Questions Conclusion List of Guidelines 109 113 125 126 Chapter How to Write Open- and Closed-Ended Questions 127 Guidelines for Writing Open-Ended Questions General Guidelines for Writing All Types of Closed-Ended Questions Guidelines for Nominal Closed-Ended Questions Guidelines for Ordinal Closed-Ended Questions Conclusion List of Guidelines Chapter Aural Versus Visual Design of Questions and Questionnaires The Importance of Visual Design in Self-Administered Surveys Visual Design Concepts and Their Application to Surveys General Guidelines for the Visual Presentation of Survey Questions Guidelines for the Visual Presentation of Open-Ended Questions Guidelines for the Visual Presentation of Closed-Ended Questions Guidelines for the Visual Presentation of Questionnaire Pages or Screens A Case Study: The Use of Visual Design Principles to Improve Data Quality in the American Community Survey Conclusion List of Guidelines 95 128 134 142 150 164 167 169 172 173 183 192 197 204 218 224 225 Chapter Ordering Questions and Testing for Question Order Effects 228 Question Order Testing Questions and Questionnaires Conclusion List of Guidelines 229 241 256 257 www.it-ebooks.info Dillman 496 bindauth.tex V2 - 07/10/2014 Author Index Willimack, D K., 48, 210, 413, 462 Willis, G B., 244 Willits, F., 367, 369 Willits, F K., 154, 200, 239 Wilson, C E., 169 Wilson, S., 381 Wine, J S., 411 Wittchen, H.-U., 232 Witt-Swanson, L., 204 Wood, H., 35, 48, 351, 352, 416, 424, 425, 427 Wooten, K., 66 Wygant, S., 331 Xu, M., 292 www.it-ebooks.info Yancey, T., 66 Ye, C., 31, 331, 414 Yechiam, E., 330 Yuan, Y., 68 Zhang, C., 131, 213–216 Zhao, S., 232 Zhou, Q., 252 Zickuhr, K., 61 9:01 A.M Page 496 Dillman bindsub.tex V3 - 07/10/2014 1:34 P.M Page 497 Subject Index AAPOR’s 2010 Cell Phone Task Force, 277 2010 AAPOR Cell Phone Task Force Report, 261 AAPOR’s standard definitions, 290 Access code, 340 Accessibility Initiative (n.d.), 309 Active listening, 280 Adaptive designs, 467 Address-based sampling (ABS), 57, 65 Administrative data, 460 Administrative records, 459–461 Adult-to-adult communication style, 23 Agree/disagree scales, 155 American Association for Public Opinion Research (AAPOR), 58, 306, 393 American Association for Public Opinion Research’s Blue Book, 261 American Community Survey, 1, 33, 88, 107 American National Election Study, 65 Anchoring, 236 Anchoring response option order effect, 148 Android, 310 Android phones, 456 Animated interviewing agents, 326 Answer spaces, 319 Apple iPad, 456 Apple’s iOS, 310 Archimedes, 463 Area probability samples, 63 Arrow method, 144 Assimilation effect, 234 Association of Academic Survey Research Organizations (AAPOR), 261, 306 Attentive processing, 174 Audio-video web components, 326 Aural communication, 169, 265 Autodialer, 284, 287 Automatic calculation tools, 130, 324 Automatic login, 340 Avatars, 326 Back button, 320 Batches, 341 Behavioral Risk Factor Surveillance System (BRFSS), 262 Big data, 459–461 Bipolar scale, 134, 151 Birthday respondent selection methods, 85 Blue ink signatures, 367 Booklet formats, 355 Bottom-up processing, 174 Bounced e-mails, 345 Branching, 161 Branching instructions, 359 Break-offs, 295 Broadband Internet access, 301 Browsers, 344, 345 Bureau of Labor Statistics, 401, 457 Business reply envelopes, 372 Business surveys, 48, 286, 451, 461–462 Calculating sampling error design effect, 91 finite population correction, 90 margin of sampling error, 89, 91 sampling error formula, 91 simple random sample, 90 Call center, 261 Call tracking forms, 288 Caller ID, 290 Calling protocols, 261 Carryover effect, 235 Cascading style sheets (CSS), 304 Cash incentives, 370 CATI software system, 140 Cell phone, 301 vs landline phone, 259 Cell-only households, 258 Census, 56 blocks, 64 geography, 64 survey, tracts, 64 Census Bureau, Center for Epidemiological Studies Depression Scale (CES-D), 96 Centers for Disease Control, 262 Certified mail, 381 Character limit, 347 Check boxes, 140 Check-all-that-apply format, 266 497 www.it-ebooks.info Dillman 498 bindsub.tex V3 - 07/10/2014 Subject Index Check-all-that-apply questions, 148–150 Clickable link, 341 Client-side paradata, 327 Client-side scripting, 309, 314, 325–350 Closed-ended questions, 134 answer categories, 135 mutually exclusive categories, 137–139 positive and negative sides, 135 web surveys, 140–142 Closing screen, 317 Closure principle, 180 Cognitive burden, 169, 171 Cognitive dissonance theory, 23 Cognitive interviewing, 243 Cognitive interviews, 342, 343, 390 Cognitive-based order effect, 234 Colored backgrounds, 353 Commission errors, 361 Common fate principle, 180 Computer-assisted personal interviewing (CAPI), 84, 324 Computer-assisted telephone interviewing (CATI) system, 50, 84, 264, 324 Computerized delivery sequence file (CDSF), 57 Conceptual midpoint, 201 Construct-specific labels, 154 Construct validity, Consumer Expenditure Survey, 33, 457 Contact personalization letterhead, 367 request for help, 367 salutation, 367, 368 within-household selection method, 368 Contact rate, 259 Contrast effect, 234 Cooperation rate, 259 Cost–benefit theory, 24 Council of American Survey Research Organizations (CASRO), 70 Counter, 326 Courier delivery, 381 Cover letter, 383 Coverage, 56 analysis, 74 considerations, 74 coverage error Coverage error, 3, 57 district-wide survey, 73 include people name, 72 investigation, 73 landline telephone, maintenance and update, 73, 74 members of survey population, 72 other frame information, 74 target population, 71 telephone survey, undercoverage, Cross-national surveys, 463 Cross-sectional surveys, 455 Current Employment Statistics (CES), 401 Current Population Survey, 1, 44, 88 Custom-designed error page, 322 Customer behavior, 462 Customer satisfaction surveys, 451 Database, 344, 345 Data entry, 354 Data security, 348 confidentiality, 461 Internet surveys, 461 threats, 461 Data validation steps, 324 www.it-ebooks.info Date questions, 180–183 Decennial Census, 2000 Decennial Census, 249 2010 Decennial Census, 208, 265 Decomposing, 161 Delivery sequence file (CDSF), 57 Department of Agriculture Agricultural Resource Management Survey, 208 Descriptive open-ended questions, 128, 132 Design effect, 58 Desktop, 303 Detection method, 361 Devices, 344, 345 Diaries, 451 Disposition codes, 290, 393 Disproportionate sampling, 76 Do Not Call Registry, 293 Don’t know response option, 135, 154, 200 Double entry, 354 Drag-and-drop method, 144 Drop-down menus, 141, 324 Dropouts, 295 E-mail, 12 bulk e-mail, 330 invitation, 337 personalization, 329–330 sender name and address, 338 spam, 329, 331, 338–339 spam filters, 339 subject line, 43, 338 Economic exchange, 370 Electronic devices, 460 Electronic gift certificates, 330 Embedded e-mail or text/SMS surveys, 305 Empowered sample members, 465 1:34 P.M Page 498 Dillman bindsub.tex V3 - 07/10/2014 Subject Index Error message, 321 Establishment surveys, 286, 324, 461 European Social Survey, 464 Evaluating early completes, 346, 348 Experimental evaluations, 342, 344 Expert review, 342, 344 Face-to-face surveys, 62 Federal Express, 381 Federal Reserve, Federal Reserve Board, 462 Field period, 284 Figure/ground orientation process, 174 Fillable PDFs, 10, 305 Fills, 324 Final reminder, 373, 379, 382 Firewalls, 348 Follow-up questions, 232 Follow-up reminder, 378–379 Font, 173, 186–187 common fonts, 187 monospace, 187 proportionally spaced, 187 sans serif fonts, 186–187 script fonts, 186 serif fonts, 186–187 size, 186–187, 389 Forced-choice questions, 148–150 Four cornerstones, Foveal view, 360 Frequently asked questions (FAQs), 274, 275 Front and back cover pages banner, 363 implementation materials, 363 sponsor, 363 survey title, 363 Fully labeled scales, 158 Gallup, 262 Gallup World Poll, 464 Gamification theory, 24 Gaming theory, 29 General Social Survey, 65 Generic error messages, 322 Gestalt psychology principles, 173, 176–181 closure, 180, 181 common fate, 180, 181 common region, 180, 181, 208 continuity, 180, 181 elemental connectedness, 180, 181 pragnanz, 206 proximity, 184–185 similarity, 180, 181, 185 Gift cards, 330 Gifts/material incentives, 30 Google Consumer Surveys, 262 Graphics, 173 Hard refusals, 285 Header and footer regions, 319 Holistic approach to design, 164, 167, 450 Holistic design, 169 Hosting server, 342 Hover, 216 HTML tables, 307 Hypertext markup language (HTML), 303 Identification (ID) number, 389 Implementation protocol, 389 In-person interviews, 465 Inbound calls, 292 Incentives cost savings, 45, 345 electronic, 330–331, 345, 423 www.it-ebooks.info 1:34 P.M Page 499 499 gifts, 30, 31 interviewer incentives, 103 lottery/prize drawing, 331, 371 material, 30–31, 330, 331, 370, 371, 390 monetary, 31 nonresponse error/bias, 6, 49, 368, 402 placement in mailing, 386–387 post/postpaid/post-survey/ post payment/ contingent/promised, 30, 31, 294, 330, 369, 421 prepaid/advance, 30, 31, 40, 330, 331, 351, 368, 369, 419, 421 returned incentives, 391, 393–394 second incentive, 370, 379, 423–424, 435 size/amount, 40, 370 stuffing, 388, 392 token cash, 14, 30, 31, 45, 371, 390, 402, 421–423, 466 token of appreciation, 40, 368–371 tracking, 343, 345 Income, 265 Income question, 137 Influence theory, 23 Institutional review boards (IRBs), 31, 278, 321, 461 Instructions, 319 Item-specific error messages, 322 Inter-University Consortium for Political and Social Research (ICPSR), 96 Interactive features, 314 Interactive voice response (IVR), 10, 263 Interactive web features, 324 Dillman 500 bindsub.tex V3 - 07/10/2014 Subject Index Interest-getting questions, 231 Internal Do Not Call, 293 Internet access, 10 Internet browser, 303 Internet literacy, 427, 432 Internet panels, 452, 455 Internet protocol (IP) address, 339 Internet surveys, 61, 420 Interviewer bias, 101–103 Interviewer monitoring, 296 Interviewer procedures, 261 Interviewer protected categories, 274 Interviewer training, 261 Interviewer variance, 103 Interviewer workstation, 295 Interviewer-administered surveys, 99, 170 Introductory script, 278 Invitation letter, 374–375 iPhones, 456 iPOLL Databank, 96 Item-specific error messages, 323, 324 JavaScript, 310, 325 Junk mail, 384 Keypunching, 353 Kish method, 84 Landline phones call blocking, 258 caller ID, 258 voice mail messages, 258 Landline telephone, Leader dots, 216 Legitimate surveys, 38 Leibniz Institute for the Social Sciences in Germany, 453 Leverage-saliency theory, 24 Lewiston and Clarkston Quality of Life Survey, 406, 426 Library instruction, 328 Links to definitions, 324 List of items, 128 List-style open-ended items, 129 Longitudinal Internet Studies for the Social Sciences (LISS) panel, 452 Longitudinal surveys, 324 conditioning, 454 satisfice, 455 sponsors, 454 straight-line answers, 455 Lotteries, 371 Low construct validity See Specification error Low response rates, 259 Mail-first methodology, 434 Mail implementation, 388 Mailing costs, 385 Mail merge, 392 Mail-out dates, 383 Mail-out packages, 386 Mail questionnaires booklet formats, 355 contrast, 359 cover pages, 361–363 early returns, 395–396 mail surveys, 351 pagination, 359 paper surveys, 352 proximity, 359 question layout, 356–359 size, 359 symbols, 359 undeliverable mail, 392–393 Mail surveys, 12, 351, 353 appreciation token, 368–371 assembling, 386 batches, 388 contacts personalization, 366–368 www.it-ebooks.info envelope color and materials, 384 fold, 386 implementation protocol, 389 keypunching, 353 mailing materials size and weight, 385 mail-out dates, 383 multiple contacts, 372–373 optical scanning, 353, 354 postage paid return envelope, 371–372 postal regulations, 388 quality control and testing cognitive interviews, 390 experimental evaluations, 390 expert review, 390 fielding period, 392 mail merge, 392 pilot studies, 391 printing aspects, 392 respondent inquiries, 394–395 response rates, 366 returned incentives, 393 scheduling, 382 Manual login, 340 Margin of error, 5, 58, 90 Marketing materials, 384 Material incentives, 370 Measurement error, acquiescence, figure/ground clarity, law of Pragnanz, norm of evenhandedness, primacy/recency, reliability, response bias, response variance, social desirability, Michigan/Reuters Survey of Consumer Sentiment, 262 Midpoint, 154 1:34 P.M Page 500 Dillman bindsub.tex V3 - 07/10/2014 Subject Index Mixed-mode survey, 12, 21, 171, 357, 396, 450, 459 design, 15 paper questionnaire, 14 preelection surveys, 15 sample frame, 13 single-mode survey, 13 target population, 13 token cash incentive, 14 types, 14 Mixed-mode survey designs alternative response modes email augmentation, 428 mail-first methodology, 434 mail questionnaire, 429 reduce survey costs, 432, 433 respondent mode preferences, 424 simultaneous/concurrent, 427 software systems, 434 survey sponsorship, 432 web-first design, 429, 430 web-first methodology, 434 web-first sequential designs, 430 contact letter, 438 contact modes, 403 final contact letter, 440 improve response rates, 402 improve timeliness, 401 lower costs, 401 multiple contact modes electronic incentive, 423 obtain contact information, 418 response rates, 421 second cash incentive, 423 survey request, 418–420 synergy between contacts by multiple modes, 417 token cash incentive, 421 paper questionnaire, 441 postal mail implementation strategy, 435 questionnaire design aural and visual modes, 415 automated calculation tool, 411 conversational cues, 412 drop-down menus, 411 embedded links to definitions, 411 extremeness in aural modes, 415 general public surveying, 416 grid format, 412 mixed-mode data collection, 413 page-by-page construction, 410 research base, 416 same question format and wording, 404, 406 scalar questions, 414 scrolling web questionnaire, 409 social norms, 415 unified mode design, 412, 413 unified mode questionnaire construction, 411 reduce coverage error, 401 reduce measurement error, 402 reduce nonresponse error, 402 response modes, 403 www.it-ebooks.info 1:34 P.M Page 501 501 testing guidelines document methodological procedures, 447 initial pilot test, 447 review and test questionnaires, 445 test coordination, 446 web questions, 443 Mobile devices, 141, 453 Consumer Expenditure Survey, 457 iPads, 456 small screens, 456 smartphone, 456, 457 text messages, 456 Mode choice, 36 Monitoring progress, 346, 348 Motivated underreporting, 232 Motivational statement, 131 Multiple contacts, 331 Multiple modes, 21 Multiple-answer questions, 141, 148 Multistage sampling, 76 Mutually exclusive categories, 136–139 National Agricultural Statistics Service, 433 National Center for Education Statistics, 9, 62 National Health Interview Survey, 61, 88, 399 National Health Interview Survey on Disability, 234 National Household Education Survey, 9, 65 National Opinion Research Center (NORC), 194, 290 National Research Council, 421, 457 Dillman 502 bindsub.tex V3 - 07/10/2014 Subject Index National Science Foundation (NSF), 401 Survey of Earned Doctorates, 241 National Security Administration, 41 National Survey of College Graduates (NSCG), 454 Natural metric, 151 Navigation buttons, 319, 320 Navigational path, 361 Nebraska Annual Social Indicators Survey (NASIS), 425 Nebraska Quality of Life surveys, 430 Neustar, 288 Neutral feedback, 282 Next button, 320 Nielsen’s TV diary survey, 65 No opinion response option, 135, 200 Nominal closed-ended questions, 134, 142 forced-choice questions, 148–150 order effects, 147–148 ranking questions, 142–145 unordered categories, 146 Nominal questions, 267 Nondirective probes, 132–134 Nonmonetary incentives, 370 Nonprobability online survey, 303 Nonprobability sampling, 70, 91, 92, 452, 455–456 Nonresponse error, contact, cooperation, quality, respondent-friendly questionnaires, response rates, Nonsubstantive, 272 option, 135 response, 154 Normative-based order effect, 234 Norm of Evenhandedness, 238 Not sure response option, 200 NSF National Survey of College Graduates, 433 Number portability, 286 Numbers, 173 Numerical open-ended questions, 129 Numerical response, 128 Office of Management and Budget, 278 Omission errors, 361 Open-ended questions, 354 descriptive, 128 list-style, 192 motivational statement, 131 nondirective probes, 132–134 numerical, 130 response types, 128–129 Operating systems, 310 Optical imaging, 353 Optical scanning, 353 Opt-in panels, 303 Order effects question order effects anchoring, 236–237 appearing moderate, 239–241 carryover, 235 cognitive-based, 235, 239 consistency, 210–211, 238–239, 319 general/specific, 239, 243 norm of even handedness, 238 normative-based, 235, 239 www.it-ebooks.info priming, 234 randomized order, 240–241 subtraction, 237–238 visual design, 172, 192, 239–240, 264 response option order effects anchoring, 148 and mode, 104–105 cognitive elaboration model, 104–105 primacy, 104–105, 147–148, 415 randomized order, 146–148 recency, 104–105, 147–148, 415 subtraction, 146–147 Ordinal closed-ended questions, 134, 150, 151 balanced scales, 158 challenges, 163 construct-specific scales, 155 fully labeled and polar-point labeled scales, 159 natural metrics, 156 scale length, 153–154 unipolar and bipolar, 151 Organic data, Page-by-page design, 313 Page-by-page web survey, 357 Panel conditioning, 455 Panels and longitudinal surveys, 451 Panels surveys, 70 GfK KnowledgePanel, 452 hard-to-reach populations, 453 Internet panels, 452 LISS panel, 452 Nielsen, 453 Paper questionnaires, 14, 352 1:34 P.M Page 502 Dillman bindsub.tex V3 - 07/10/2014 Subject Index Paper surveys, 352 Paradata, 296, 313, 327, 328 Paralinguistic cues, 280 Paralinguistic features, interviewer’s voice, 170 People’s reluctance See also Social exchange; Survey participation benefits critical questions, 19 multiple modes, 21 phone calls follow-up, 20 respondent-friendly questionnaires, 21 response rate, 21–23 Pew Internet & American Life Project, 301 Pew Research Center, 259, 262, 454 Pilot studies, 342, 343, 391 Polar-point scales, 159 Portable People Meter (PPM), 458 Post incentives, 369 Postage, 372 Postage paid return envelope, 371–372 Postal regulations, 388 Postal returns, 392 Postal surveys, 12 Postpaid contingent incentive, 30 Poststratification, 58 Postsurvey adjustments base weight, 87 nonresponse adjustments, 88 poststratification weighting/raking, 88 probability of selection, 87 weighting, 89 Potentially objectionable questions, 231 Pragnanz, 206 Preattentive processing, 174 Preelection surveys, 15 Prenotice letter, 373 Prepaid cash incentive, 331 Prepaid token incentive, 30, 368 Prevention method, 360 Primacy, 104, 147 Priming, 234 Priority mail, 381 Prize drawings, 371 Probability sample survey, Probability sampling, 452, 455 clustering, 75 disproportionate sampling, 76 finite population correction, 79 large populations, 80 margin of error, 82 multistage sampling, 76 proportionate sampling, 76 sample size, 77 sampling error, 78 simple random sampling, 75 small populations, 81 stratification, 76 subgroups of populations, 83 within-household selection, 83, 87 Probe, 132–134 Programming survey-specific messages, 322 Progress indicator, 325 Progress monitoring website, 347 Promised/contingent incentives, 369 Proportionate sampling, 76 Proximity principle, 178 Public opinion surveys, 262 QR codes, 303 Quality of Life in a Changing Nebraska (QLCN) survey, 407, 425 Quasiprobability method, 85 www.it-ebooks.info 1:34 P.M Page 503 503 Question layout arrangement, 359 grouping, 357 legal-size folded format, 356 one-column format, 356 two columns format, 356 Question order, 229 anchoring, 236 carryover effect, 235 consistency, 239 events, 233 filter questions, 232 interest-getting questions, 231 moderate, 239 norm of evenhandedness, 238 potentially objectionable questions, 232 priming, 234 related question grouping, 230 subtraction, 238 Question stem, 319 Questionnaire, 19, 261 covers, 363, 383 design, 302 mailing, 373 Quick response codes, 304 Quota sampling, 85 Radio buttons, 140 Random digit dial (RDD) sampling, 66, 67 telephone surveys, 9, 47, 68 Ranking questions, 142–145 Reasoned action theory, 23 Recency, 104, 147 Reciprocal obligation, 371 Reference point effects, 156 Refusal conversion, 285 Reinterview surveys, 298 Replacement questionnaire mailing, 367, 373 Research Triangle Institute (RTI), 411 Dillman 504 bindsub.tex V3 - 07/10/2014 Subject Index Respondent confusion, 265 Respondent inquiries, 346 Respondent-friendly questionnaires, Respondent region, 319 Response option order, 148 Response options, 319 Response process/survey response process, 107–109, 169, 176 Response rate, 58, 259, 399 Response rate calculation (AAPOR) RR3, 58 Retrospective interviewing technique, 247 Returned incentives, 393 Return envelope, 386 Rizzo method, 84 Roper Center for Public Opinion Research, 96 Sample survey, 2, 56 Sampling error, 3–5, 56–57, 59 Sampling error formula, 91 Sampling frames, 57 address-based sampling (ABS) American National Election Study, 65 CDSF, 65 General Social Survey, 65 geocoding, 65 geographic stratification, 65 GfK Knowledge Network’s KnowledgePanel, 65 GPS coordinates, 65 in-person survey, 66 National Household Education Survey, 65 Nielsen’s TV diary survey, 65 USPS, 65 area probability samples census blocks, 64 census geography, 64 census tracts, 64 housing unit, 64 U.S Postal Service, 64 CDSF, 70, 71 Internet frame, 69, 70 RDD sampling attempts, 68 calling protocols, 68 calling schedules, 68 dual-frame RDD sampling, 67 geographic areas, 68 landline frame, 67 list-assisted RDD methods, 67 telephone directories, 69 Satisfaction survey, 462–463 Satisfice, 136 Screen layouts, 320 Screen resolution, 307 Screenshots, 255 Scrollable web survey design, 313, 314 Second incentive, 370, 423 Section 508 of the Rehabilitation Act, 308 Secure Socket Layering (SSL), 348 Self-administered modes, 271 Self-administered survey, 170 Sequential mixed-mode designs, 416 Server-side paradata, 327 Server-side scripting, 309 Servers, 304 Short field period, 262 Simple random sampling, 75 Single-mode methods, 400 Single-mode surveys, 13, 398, 450 Internet, 258 mail surveys, 258 telephone, 258 www.it-ebooks.info Skip instructions, 360 Slider scale, 140 Smartphone ownership, 301 Smartphones, 10, 11, 141, 456 Social desirability, 99, 100, 102 Social exchange, 16, 23, 329, 368, 450 achieve self-interests, 25 adult-to-adult communication style, 23 asynchronous, 26, 27 benefits, 21 cognitive dissonance theory, 23 cost–benefit theory, 24 costs, 21 costs of participation business reply, 35 foot-in-the-door technique, 37 hard-to-reach populations, 35 lower response rates, 36 mode preference, 36 panel surveys, 37 reduce complexity, 33 reduce length, 32, 33 researcher-centric strategy, 34 sensitive information, 36 stamped return envelope, 35 subordinating language, 34 telephone, 32 visual design principles, 34 economic exchange, 25 establishing trust confidentiality, 40 design communications, 41 fake surveys, 38 Gallup organization, 40 1:34 P.M Page 504 Dillman bindsub.tex V3 - 07/10/2014 Subject Index government-sponsored surveys, 39 information security, 40 legitimate surveys, 38 preexisting relationship, 40 private organization sponsorship, 39 token incentives, 40 university sponsorship, 39 gamification theory, 24 guidelines confidentiality, 44 consent form, 46 envelopes/subject lines, 43 holistic design, 43 longitudinal survey, 44 multiple contacts, 44 tailored designs, 43 URL, 43 influence theory, 23 leverage-saliency theory, 24 mail survey, 25 mixed-mode designs adaptive design, 49 business surveys, 48 multiple modes of contact, 47, 403 multiple modes of response, 48, 403 responsive design, 49 rapid-fire communications, 26 rational behavior model, 25 reasoned action theory, 23 self-administered web, 25 social interaction, 26 technology, 27 trust, 21 WSU Doctoral Experience Survey email augmentation, 50 initial communications, 51 interest-getting questions, 51 paper questionnaire, 52 token of appreciation, 50 Social media posts, 459 Soft refusals, 285 Specification error, Sponsors, Stamped return envelopes, 372 Standardized interview, 278 State laws, 288 Statistics Netherlands, 452 Stratification, 57, 76 Strong satisficing, 109 Student surveys, 127 Subordinating language, 34 Subtraction order effects, 146 Supplementing questionnaires, 458, 459 Survey apps, 305, 310 Survey benefits, 451 Survey contacts personalization, 366 Survey conversation, 170 Survey costs, 451 Survey coverage in-person surveys, 58, 62 Internet, 61 landline/mobile phone, 60 language and literacy issues, 62 Survey culture, 451 Survey error coverage error, 3, 4, 57–59, 71–74, 401–402 measurement error, 3–4, 6–8, 170, 321, 402–403, 414 nonresponse error, 3, 5–6, 48–50, 402 sampling error, 3–5, 58, 59, 75–78, 87–93 specification error/ construct validity, www.it-ebooks.info 1:34 P.M Page 505 505 Survey modes ACS, 107 aural vs visual communication cognitive elaboration model, 104 intervieweradministered surveys, 105 paralinguistic features, 104 primacy, 104, 105 recency, 104, 105 telephone interviews, 104 visual design elements, 104 working memory, 104 computerization errors of commission, 106 errors of omission, 106 fills, 106 paper surveys, 106 skip patterns, 106 testing phase, 106 validation, 106 presence vs absence of interviewers acquiescence, 100 agree/disagree types of questions, 100 clustering, 103 customer satisfaction survey, 99 interviewer bias, 101, 103 interviewer experience, 101 interviewer gender, 101 interviewer monitoring, 103 interviewer training, 103 interviewer variance, 103 intraclass correlation (rho), 103 item nonresponse rates, 102 Dillman 506 bindsub.tex V3 - 07/10/2014 Subject Index Survey modes (continued) lack of standardization, 103 monitor interviewer performance, 104 nonresponse error, 104 normative question order effects, 101 pace of survey, 99 question order, 99 questions/probe, 99 race of interviewer effects, 101 sample variance, 103 self-administered surveys, 99 sensitive questions, 100 skip patterns, 99 social desirability, 99, 100, 102 social norms, 99, 100 standardize the questionnaire, 104 telephone respondents, 100 train interviewers, 104 variance, 103 respondents comprehension stage, 109 income, 107 judgment and report, 109 motivation, 107 optimizing, 109 perception, 108 response process/survey response process, 107–109 retrieve relevant information, 109 satisfice, 109 sensitive information, 107 sensitive questions, 107 strong satisficing, 109 weak satisficing, 109 time-series data, 107 weighting, 107 Survey Monkey, 262 Survey participation benefits, 27 additive effects, 31 advice/assistance, 28 beneficiaries, 27 gaming theory, 29 gifts/material incentives, 30 interesting questions, 28 lotteries, 31 opportunities to respond, 30 postpaid contingent incentive, 30 prepaid token incentive, 30 survey results, 28 token cash incentives, 30 utilize sponsorship, 29 Survey sponsors, 19 Survey technology, 451 Survey trust, 451 Survey weights, 58 Survey-specific error message, 322, 323 Symbols, 173 Tablet ownership, 301 Tablets, 10, 11, 141, 302 Tailored design, 15, 16, 450, 451 Target population, 2, 13, 57 Task-related feedback, 282 Telephone call attempts duration between attempts, 285 legal limitations, 285 number of attempts, 285 refusal conversion, 285 rules, 285 Telephone Consumer Protection Act (TCPA), 288 Telephone directories, 69 Telephone revision, 272 Telephone surveys, 4, 12 abbreviations, 273 active listening, 280 www.it-ebooks.info branching, 280 call attempts, 286 caller ID, 290 calling procedures, 259, 284 calling rules, 261 calls with minors, 276 complex instructions, 265 contact and cooperation rates, 259 continuous feedback and training, 283 design, 277 directive probes, 283 displaying question, 275 disposition, 288 Do Not Call (DNC) list, 293 Don’t Know option, 273 experienced interviewers, 283 FAQs, 274 fills, 280 filtering, 280 follow-up training, 282 geographic information, 276 ID number, 290 inbound calls, 292 interviewers, 262, 270 interviewers vs respondents, 273 manual/physical dial, 288 monitoring and feedback, 261, 283 neutral feedback, 282 noncontact, 289 nonsubstantive response options, 274 order of the questions, 279 pace, 281 paired comparison, 267 pitch, 281 probing, 282 quality control and testing cognitive interviews, 294 1:34 P.M Page 506 Dillman bindsub.tex V3 - 07/10/2014 Subject Index experimental evaluations, 294 expert review, 294 interviews validation, 297 monitoring interviewer, 296 monitoring progress, 295 paradata, 296 pilot studies, 294 programming, 295 ranking, 266 record verbatim response, 268 refused option, 273 respondent cooperation, 281 respondents, 267 screen, 277 self-administered surveys, 263 specific script, 274 standardized scripts, 275 supervised practice, 283 surveys types, 262 task-related feedback, 282 telephone number, 284–285 time pressures, 278 tone, 281 training, 283 voice mail, 292 Testing/pretesting cognitive interviews, 243–249, 294 limits/shortcomings, 248 in mail surveys, 390–391 reporting procedures, 248 retrospective interviewing, 247 in telephone surveys, 294 think-aloud, 244 in web surveys, 342–344 document and archive, 255–256 experimental evaluations, 249–250 in mail surveys, 390–391 in telephone surveys, 294 in web surveys, 342–344 expert reviews, 242–243 in mail surveys, 390–391 in telephone surveys, 294 in web surveys, 342–344 eye tracking, 252–254 in mail surveys, 254 in telephone surveys, 254 in web surveys, 252–254 focus group, 249 mixed-mode surveys, 445–447 mode/modes, 244, 254–255 pilot study, 251–252 in mail surveys, 390–391 in telephone surveys, 294 in web surveys, 342–344 usability study, 249 Text messages, 301, 456 Thank-you postcard, 373 Thank-you reminder, 375–377 Think-aloud cognitive interviews, 244 Title 13, 33 Token cash incentives, 30, 421 Token incentives, 368, 370 Top-down processing, 174 www.it-ebooks.info 1:34 P.M Page 507 507 Total Survey Error (TSE), framework, testing, 10 Touchtone data entry, 10 U.S Postal Service (USPS), 12, 65 Undecided response option, 135, 154, 200 Undeliverable mails, 388, 392–393 Unified mode construction, 450 Unipolar scale, 135, 151 Unique identification number, 290, 339 United Parcel Service (UPS), 381 United States Department of Agriculture Survey of Farm Operators, 176 University of Georgia, 133 University of Nebraska–Lincoln (UNL), 417, 432 URL, 303 Usability, 306 User agent string, 310 Vague quantifiers, 151 Validation study, 298 Vision sciences, 173 Visual analog scales, 140, 324 Visual communication, 169 Visual cues, 169 Visual design, 171, 319 background colors, 207 closed-ended questions, 198 answer spaces, 203 mobile surveys, 199 nonsubstantive response options, 201 ordinal, 199 consistency, 206 Dillman 508 bindsub.tex V3 - 07/10/2014 Subject Index Visual design (continued) contrast and enclosure, 208 elements, 173 font size and line length, 187 grid and matrix formats, 215, 216 grouping and subgrouping, 205 item nonresponse, 172 legibility, 186 measurement error, 172 minimize respondent burden, 34 navigational path, 188 open-ended questions, 192 answer spaces, 194 descriptive, 192 list-style, 193 numeric, 195 templates, 196–197 page layout and navigational path, 204 properties, 174 question layout, 190 question numbers, 210 report date information, 182 reverse print, 185 skip patterns, 172 subgrouping, 184 U.S Census questionnaires, 34 usability, 34 visual clutter reduction, 213 visual processing basic building blocks, 174 basic page layout, 174 Gestalt grouping principles, 176–181 information organization, 174 task completion, 176 white answer spaces, 211 Visual layout, 320 Visual midpoint, 201 Voice mail, 292 Volunteered categories, 274 Washington and Pennsylvania Electricity studies, 430 Washington Community surveys, 430 Washington Economic Survey, 57 Washington State University (WSU), 21, 417 Washington/Nebraska water surveys, 363, 432, 435 Weak satisficing, 109 Web panels, 401 Web server, 341 Web surveys, 19, 410 answer spaces, 203 browser-based web survey, 305 coverage error, 401 cumulative response rate, 22 data security, 461 drop-down menu format, 141 forced-choice question format, 150 graphical progress indicator, 325 implementation, 328–329 mixed-mode designs, 303 mobile devices, 62, 456 multiple contacts, 331 nonprobability sampling, 91–92 page-by-page web survey, 312, 413 paradata, 327 probing, 133 quality control and testing document and archive, 255–256 cognitive interviews, 342–344 www.it-ebooks.info experimental evaluations, 342–344 expert review, 342–344 eye tracking, 252–254 pilot studies, 342–344 programming, 295 ranking questions, 144–145 regions, 209 screen-by-screen design, 410 screen size and resolution, 308 scrolling web survey, 314 unique identification number, 339–341 usability study, 249 visual analog scale, 140–141 visual design, 173, 185 Web-first mixed-mode designs, 430, 434 Website accessibility, 308 Weighting error, 58 Welcome or opening screen, 315 White answer spaces, 353 Within-household selection, 384 Words, 173 World Wide Web Consortium (W3C) Web, 309 Writing survey questions concepts established measures, 96 goals, 95 preexisting measures, 96 write research questions/statements, 95 guidelines ask one question at a time, 116 avoid double-or triple-barreled questions, 116 1:34 P.M Page 508 Dillman bindsub.tex V3 - 07/10/2014 Subject Index avoid double negatives, 122 complete sentences, 121 concrete words, 119 make easier to comprehend, 123 question formats, 113 self-administered survey, 124 simple and familiar words, 117 simple questions, 120 technical accuracy, 117 visual analog scales, 114 information types attitudes and opinions, 97 behaviors and events, 98 factual/demographic information, 97 question formats additional instructions, 109 answer spaces/choices, 109 choosing the format, 113–114 closed-ended question formats, 110, 112 mobile devices, 110 nominal questions, 112 open-ended question formats, 110 ordinal questions, 112 www.it-ebooks.info 1:34 P.M Page 509 509 partially closed-ended question format, 113 question stem, 109 scales, 112 visual cues, 112 Writing survey questions survey mode, 98 WSU Doctoral Student Experience Survey, 22, 416, 417 WSU Student Experience Surveys, 138, 401 YouGov, 262 Z fold, 387 WILEY END USER LICENSE AGREEMENT Go to www.wiley.com/go/eula to access Wiley’s ebook EULA www.it-ebooks.info ... readers in this fourth edition of Internet, Phone, Mail, and Mixed- Mode Surveys There were issues we could not address in the pages of the book because of space limitations and the constraints of... ffirs1.tex V3 - 06/30/2014 9:42 A.M Page ii Dillman ffirs1.tex V3 - 06/30/2014 FOURTH EDITION INTERNET, PHONE, MAIL, AND MIXED- MODE SURVEYS The Tailored Design Method Don A Dillman Jolene D Smyth Leah Melani... ffirs1.tex V3 - 06/30/2014 9:42 A.M Page ii Dillman ffirs1.tex V3 - 06/30/2014 FOURTH EDITION INTERNET, PHONE, MAIL, AND MIXED- MODE SURVEYS www.it-ebooks.info 9:42 A.M Page i Dillman www.it-ebooks.info