Web Survey Bibliography
Relevance and Research Question:
It is well-documented that online surveys elicit higher reports of socially undesirable behavior than interviewer-administered surveys. However, there are possible exceptions, where the form of the question may inhibit the revelation of prejudicial attitudes. In research exploring race of interviewer effects, Krysan and Couper (2003) found some instances where white respondents (for example) gave more negative responses to interviewers than to computerized instruments. In qualitative debriefings, some respondents noted that talking to an interviewer gave them an opportunity to explain their choice of responses; in the CASI condition (as on the Web; see Krysan and Couper, 2005), they could only pick one of the response options, without the opportunity to justify their choice. We called this the “I’m not a racist, but…” phenomenon. An online experiment was designed to explore the hypothesis that, when given an opportunity to explain or clarify their answers, respondents will give more prejudicial responses.
Methods and Data:
Two experiments were embedded in the LISS online probability-based panel in the Netherlands. In both cases a set of 9 items on attitudes toward immigrants was asked. In the first study, conducted in August 2009 (n=4639), a random half received an open-ended question on a separate page following each closed question. In the second study, conducted in December 2010 (n=5328), for a random half of respondents, an optional open-ended comment appeared below each closed-ended question on the same page.
The results provide support for the hypothesis. In both cases, respondents given the open question gave significantly more prejudiced responses (F[1, 4352]=25.6, p<.001 for Exp. 1 and F[1, 5326]=7.1, p=0.008 for Exp. 2) than those getting only the closed-ended question. However, contrary to expectation, the effect was larger in experiment 1 than 2. We explore this finding in greater detail, examining both responses to individual items and those who made use of the text box to offer comments.
This study suggests value in giving respondents the opportunity to voice their opinions in their own words, rather than just requiring them to agree or disagree with one of the response options.
GOR Homepage (abstract) / (presentation)
Web Survey Bibliography - Social research (725)
- The Effect of Differential Mailing Methodologies on Response Rates: Testing Advanced Notices, Pre-Recorded...; 2013; Pens, Y., Cantave, M. A., Gentry, R. J.
- Alone in a Group: Comparison of Effects of a Group-Administered Paper-Pencil Survey Versus an Individually...; 2013; Higgins, W. B., Barlas, F. M., Pflieger, J., Thomas, R. K., Jeffery, D., Mattiko, M.
- The Effect of Survey Mode on Socially Undesirable Responses to Open Ended Questions: Online vs. Paper...; 2013; Hedberg, E., Ceasar, G., Wallace, D.
- Estimation of Expected Academic Engagement Behaviors: The Use of Vague Quantifiers Versus Tallied Responses...; 2013; Cole, J.
- First Contact Strategies for Web Surveys: Is a Phone Call or a Letter the More Effective Introduction...; 2013; Connelly, J., Sjoblom, M., Datta, R., Scholfield Hepburn, P.
- The Effectiveness of Mailed Invitations for Web Surveys; 2013; Bandilla, W., Couper, M. P., Kaczmirek, L.
- One Drink or Two: Does Quantity Depicted in an Image Affect Web Survey Responses?; 2013; Charoenruk, N., Stange, M.
- Multi-Mode Survey Administration: Does Offering Multiple Modes at Once Depress Response Rates?; 2013; Newsome, J., Levin, K., Langetieg, P., Vigil, M., Sebastiani, M.
- The Digital Divide: The internet and social inequality in international perspective; 2013; Ragnedda, M., Muschert, G.
- Survey quality prediction system 2.0; 2013
- Practical tools for designing and weighting survey samples; 2013; Valliant, R. L., Daver, J. A., Kreuter, F.
- Cognitive Probes in Web Surveys: On the Effect of Different Text Box Size and Probing Exposure on Response...; 2013; Behr, D., Bandilla, W., Kaczmirek, L., Braun, M.
- Sensitive Topics in PC and Mobile Web Surveys; 2013; Mavletova, A. M., Couper, M. P.
- Comparison of psychometric properties of internet versions of the Marlowe-Crowne Social Desirability...; 2013; Vesteinsdottir, V., Reips, U. -D., Joinson, A. N., Porsdottir, F.
- Experiences from a probability-based Internet panel: Sample, recruitment and participation; 2013; Scherpenzeel, A.
- Comparison of web-based versus paper-and-pencil administration of a humor survey; 2013; Wang, C.-C., Cheng, C.-L.;, Liu, K.-S., Cheng, Y.-Y.
- The equivalence of Internet versus paper-based surveys in IT/IS adoption research in collectivistic...; 2013; Fang, J., Wen, C., Prybutok, V.
- Mode Effects in Free-list Elicitation: Comparing Oral, Written, and Web-based Data Collection; 2013; Gravlee, C. C., Bernard, H. R., R., Jacobsohn, A., R.Maxwell, C. R.
- The practice of social research; 2012; Babbie, E. R.
- The integration of facebook into class management: an exploratory study; 2012; Chou, P. N.
- The cross platform report. Q2 -2012 - US; 2012
- Sensitive topics in PC Web and mobile web surveys: Is there a difference?; 2012; Mavletova, A. M., Couper, M. P.
- Selection bias of internet panel surveys: A comparison with a paper-based survey and national governmental...; 2012; Tsuboi, S. et al.
- Research company spotlight - Mobile surveys; 2012
- Quality in market research. From theory to practice. 2nd Edition; 2012; Harding, D., Jackson, P.
- Ofcom technology tracker Wave 3; 2012
- Guide to social science data preparation. Best practice throughout the data life cycle; 2012
- Google et Médiamétrie créent une audience bimédia; 2012; Gonzales, P.
- Especially for You: Motivating Respondents in an Internet Panel by Offering Tailored Questions; 2012; Oudejans, M.
- Specialized Tools for Measuring Past Events ; 2012; Belli, R. F.
- Subjective Well-being Of Spanish Workers: Continuous Voluntary Web Survey Examination; 2012; de Pedraza, P., Guzi, M.
- Recruiting in an Internet panel using respondent driven sampling; 2012; Schonlau, M.
- Measuring Work Activities and Skill Requirements of Occupations: Experiences from a European Pilot Study...; 2012; Tijdens, K., De Ruijter, E., De Ruijter, J.
- Reaching and Hearing the Invisible: Organizational Research on Invisible Stigmatized Groups via Web...; 2012; Trau, R. N. C., Haertel, C. E. J., Haertel, G. F.
- What Survey Modes are Most Effective in Eliciting Self-Reports of Criminal or Delinquent Behavior?; 2012; Kleck, G., Roberts, K.
- An assessment of equivalence between Internet and paper-based surveys: evidence from collectivistic...; 2012; Fang, J., Wen, C., Prybutok, V.
- Item comparability in cross-national surveys: results from asking probing questions in cross-national...; 2012; Behr, D., Braun, M., Kaczmirek, L., Bandilla, W.
- Assessing Cross-National Equivalence of Measures of Xenophobia: Evidence from Probing in Web Surveys; 2012; Behr, D., Braun, M., Kaczmirek, L.
- Studying Migrants with the Help of the Internet: Methods from Psychology; 2012; Reips, U. -D., Buffardi, L.
- Panel Conditioning in Online Survey Panels: Problems of Increased Sophistication and Decreased Engagemeent...; 2012; Adams, A. N., Atkeson, L. R., Karp, J. A.
- Do Questions about Watching Internet Pornography Make People Watch Internet Pornography? A Comparison...; 2012; Peter, J., Valkenburg, P. M.
- The Propensity of Older Respondents to Participate in a General Purpose Survey; 2012; Lynn, P.
- Mode-Switch Protocols: How a Seemingly Small Design Difference can affect Attrition Rates and Attrition...; 2012; Lynn, P.
- Going online with a face-to-face household panel: Initial results from an experiment on the UK Household...; 2012; Jaeckle, A., Lynn, P., Burton, J.
- The use of new technologies on the British Birth Cohort Studies; 2012; Calderwood, L.
- Not by the Book: Facebook as a Sampling Frame; 2012; Brickman Bhutta, C.
- Design of CAWI Instruments for Social Surveys ; 2012; Blanke, K.
- Psychometric properties of an internet administered version of the Marlowe-Crowne Social Desirability...; 2012; Vesteinsdottir, V., Reips, U. -D., Joinson, A. N., Porsdottir, F.
- GESIS Online Access Panel Pilot Study: Recruitment and Panel Maintenance; 2012; Kaczmirek, L., Bandilla, W., Schaurer, I., Struminskaya, B., Weyandt, K.
- The German Internet Panel: First Results from the Recruitment Phases; 2012; Blom, A. G.