Web Survey Bibliography
Relevance and Research Question:
It is well-documented that online surveys elicit higher reports of socially undesirable behavior than interviewer-administered surveys. However, there are possible exceptions, where the form of the question may inhibit the revelation of prejudicial attitudes. In research exploring race of interviewer effects, Krysan and Couper (2003) found some instances where white respondents (for example) gave more negative responses to interviewers than to computerized instruments. In qualitative debriefings, some respondents noted that talking to an interviewer gave them an opportunity to explain their choice of responses; in the CASI condition (as on the Web; see Krysan and Couper, 2005), they could only pick one of the response options, without the opportunity to justify their choice. We called this the “I’m not a racist, but…” phenomenon. An online experiment was designed to explore the hypothesis that, when given an opportunity to explain or clarify their answers, respondents will give more prejudicial responses.
Methods and Data:
Two experiments were embedded in the LISS online probability-based panel in the Netherlands. In both cases a set of 9 items on attitudes toward immigrants was asked. In the first study, conducted in August 2009 (n=4639), a random half received an open-ended question on a separate page following each closed question. In the second study, conducted in December 2010 (n=5328), for a random half of respondents, an optional open-ended comment appeared below each closed-ended question on the same page.
The results provide support for the hypothesis. In both cases, respondents given the open question gave significantly more prejudiced responses (F[1, 4352]=25.6, p<.001 for Exp. 1 and F[1, 5326]=7.1, p=0.008 for Exp. 2) than those getting only the closed-ended question. However, contrary to expectation, the effect was larger in experiment 1 than 2. We explore this finding in greater detail, examining both responses to individual items and those who made use of the text box to offer comments.
This study suggests value in giving respondents the opportunity to voice their opinions in their own words, rather than just requiring them to agree or disagree with one of the response options.
GOR Homepage (abstract) / (presentation)
Web Survey Bibliography - USA (2403)
- The Gamification of Marketing Researc; 2013; Donato, P., Link, M. W.
- Measuring Up: Impact of mobile and segmentation on respondent behaviour; 2013; Luck, K.
- Best of Both Worlds? Can we make convenience samples representative?; 2013; Doe, P.
- Multimode, Global Scale Usage: Understanding respondent scale usage across borders and devices; 2013; Pettit, F. A., Courtright, M.
- Why Big Data is a Small Idea…and Why You Shouldn’t Worry So Much; 2013; Needel, S.
- Service with an e-smile: Employee authenticity and customer use of web-based support services; 2013; Turel, O., Connelly, C. E., Fisk, G. M.
- Survey Breakoffs in a Computer-Assisted Telephone Interview; 2013; McGonagle, K.
- Developing a New Mixed-Mode Methodology For a Provincial Park Camper Survey in British Columbia; 2013; Dyck, B. W.
- Mode effect analysis and adjustment in a split-sample mixed-mode Web/CATI survey; 2013; Kolenikov, S., Kennedy, C.
- Exploring factors associated with respondent mode choice for surveys using mobile devices.; 2013; Walton, L.
- Mobilizing your Branded Panel: Panel data quality during the smartphone transition; 2013; Kugel, C., Brien, D., Blechman, J.
- Computers, Tablet & Smart Phones: The Truth about Web‐based Surveys; 2013; Merle, P., Gearhart, S., Craig, C., Rahimi, M., Brooks, M. E., Vandyke, M.
- An Examination of the Relationship Between Pretest Method Results and Data Quality; 2013; Maitland, A.
- Associations Between Interactional Indicators of Problematic Questions and Systems for Coding Question...; 2013; Dykema, J., Schaeffer, N. C., Garbarski, D.
- Investigating the Bias of Alternative Statistical Inference Methods in Sequential Mixed-Mode Surveys; 2013; Suzer-Gurtekin, Z., Heeringa, S. G., Valliant, R. L.
- The Attention Span of a Goldfish: Impacts of Survey Mobilization on Respondents ; 2013; Luck, K.
- Adding Observation to Copy Testing with Kids; 2013; Hurley, B., Salo, M.
- Gamified Surveys: Journey Into Product Development; 2013; Adamou, B.
- Phish Rising: How Internet Criminals are Undermining the Viability of Online Survey Research…and...; 2013; Kunovic, K.
- The State of Technology in Market Research; 2013; Macer, T.
- Historic Resource Surveys and the Internet; 2013; The Center for Historic Architecture University of Houston Gerald D. Hines College of Architecture
- Effects of Sponsorship on Response: Mixed-Mode Web and Mail Surveying ; 2013; Edwards, M. L., Dillman, D. A., Smyth, J.
- Designing a Pilot to Introduce Web Collection in the UK Labour Force Survey; 2013; Merad, S.
- Data Collection Method Comparisons for the 2011 Fishing, Hunting, and Wildlife-Associated Recreation...; 2013; Herbstritt, M., Hornick, D.
- A mixed methods approach to investigate how Cognitive Interviewing findings can improve the validity...; 2013; Padilla, J. L., Miller, K., Loeb, M.
- Multiple Group Confirmatory and Exploratory Factor Analysis: Exploring Measurement Invariance by Rotational...; 2013; Asparouhov, T.
- BSEM Measurement Invariance Analysis; 2013; Muthen, B. O.
- ReCal OIR: Ordinal, Interval, and Ratio Intercoder Reliability as a Web Service; 2013; Freelon, D. G.
- Classifying Mouse Movements and Providing Help in Web Surveys; 2013; Horwitz, R.
- Video Interviewing: An Exploration of the Feasibility as a Mode of Survey Application; 2013; Jeannis, M., Terry, T. L., Heman-Ackah, R., Price, M.
- Matching Data Collection Method to Purpose: In the Moment Data Collection with Mobile Devices for Occasioned...; 2013; Shea, C., Roberts, M., Johnson, E. P., Hadlock, W.
- Filling the Void: Gaining a Better Understanding of Tablet-based Surveys; 2013; Wells, T., Bailey, J., Link, M. W.
- Old versus New: The Comparative Efficiency of Mail and Internet Surveys of State Legislators.; 2013; Fisher, S. H., Herrick, R.
- Mail and Web-Based Survey Administration: A Case Study With Recreational Users of Virginia's Wildlife...; 2013; Carrozzino-Lyon, A. L., McMullin, S. L., Parkhurst, J. A.
- Using Mixed-Mode Contacts in Client Surveys: Getting More Bang for Your Buck; 2013; Israel, G. D.
- Mode Effects in Mixed-Mode Surveys: Prevention, Diagnostics, and Adjustment 1; 2013; de Leeuw, E. D., Dillman, D. A., Schouten, B.
- Measurement Invariance; 2013; Muthen, B. O., Oberski, D., Klausch, L. T., Asparouhov, T.
- Comparing Web-based with Mail Survey Administration of the Consumer Assessment of Healthcare Providers...; 2013; Bergeson , S. C., Gray, J., Ehrmantraut, L. A., Hays, R. D.
- Data Collection in Sociolinguistics: Methods and Applications; 2013; Mallinson, C., Childs, B., Van Herk, G.
- Cultural Challenges in Online Survey Data Collection; 2013; Baron, N. S.
- Online Survey Data Quality and its Implication for Willingness-to-Pay: A Cross-Country Comparison; 2013; Gao, Z., House, L., Jing, X.
- Self-Rated Health Assessed by Web Versus Mail Modes in a Mixed Mode Survey: The Digital Divide Effect...; 2013; Shim, J.-M., Shin, E., Johnson, T. P.
- Online Survey Design and Development: A Janus-Faced Approach; 2013; Lauer, C., McLeod, M., Blythe, S.
- Survey Sidekick: Learning & designing scientifically sound surveys; 2013; Hsiao, I.-H., Malhotra, M., Joo, J., Chae, H. S., Natriello, G.
- Satisficing in Web Surveys: Implications for Data Quality and Strategies for Reduction; 2013; Zhang, C.
- The Handbook of Questionnaire Design; 2013; Krosnick, J. A., Fabrigar, L. R.
- Using Technology to Conduct Questionnaire Evaluations with Hard to Reach Populations ; 2013; Ridolfo, H., Ott, K.
- Usability and Accessibility Guidelines for Web Sites and Web Surveys ; 2013; Ashenfelter, K. T.
- Evaluation of a Sequential Mixed-Mode Design Experiment with Physicians on Response Rates, Costs, and...; 2013; Geisen, E., Olmsted, M. G., Murphy, J., Stanley, M.
- When We Do Not Know the Difference – the Level of DK in Different Question Formats and Different...; 2013; Schwarzer, S., Zeglovits, E., Connor, D. S.