Web Survey Bibliography
Relevance and Research Question:
It is well-documented that online surveys elicit higher reports of socially undesirable behavior than interviewer-administered surveys. However, there are possible exceptions, where the form of the question may inhibit the revelation of prejudicial attitudes. In research exploring race of interviewer effects, Krysan and Couper (2003) found some instances where white respondents (for example) gave more negative responses to interviewers than to computerized instruments. In qualitative debriefings, some respondents noted that talking to an interviewer gave them an opportunity to explain their choice of responses; in the CASI condition (as on the Web; see Krysan and Couper, 2005), they could only pick one of the response options, without the opportunity to justify their choice. We called this the “I’m not a racist, but…” phenomenon. An online experiment was designed to explore the hypothesis that, when given an opportunity to explain or clarify their answers, respondents will give more prejudicial responses.
Methods and Data:
Two experiments were embedded in the LISS online probability-based panel in the Netherlands. In both cases a set of 9 items on attitudes toward immigrants was asked. In the first study, conducted in August 2009 (n=4639), a random half received an open-ended question on a separate page following each closed question. In the second study, conducted in December 2010 (n=5328), for a random half of respondents, an optional open-ended comment appeared below each closed-ended question on the same page.
The results provide support for the hypothesis. In both cases, respondents given the open question gave significantly more prejudiced responses (F[1, 4352]=25.6, p<.001 for Exp. 1 and F[1, 5326]=7.1, p=0.008 for Exp. 2) than those getting only the closed-ended question. However, contrary to expectation, the effect was larger in experiment 1 than 2. We explore this finding in greater detail, examining both responses to individual items and those who made use of the text box to offer comments.
This study suggests value in giving respondents the opportunity to voice their opinions in their own words, rather than just requiring them to agree or disagree with one of the response options.
GOR Homepage (abstract) / (presentation)
Web Survey Bibliography - Internet access Panels (460)
- Effects of Lotteries on Response Behavior in Online Panels; 2013; Goeritz, A., Luthe, S. C.
- Lotteries and study results in market research online panels; 2013; Goeritz, A., Luthe, S. C.
- Ten questions to ask your online survey provider; 2013; Williams, D.
- An approach to selecting online respondents; 2013; Terhanian, G.
- By the Numbers: Theory of adaptation or survival of the fittest?; 2013; Cavallaro, K.
- Using a web-based survey tool to undertake a Delphi study: Application for nurse education research; 2013; Gill, F. J., Leslie, G. D., Grech, C., Latour, J. M.
- Does one really know?: Avoiding noninformative answers in a reliable way.; 2013; de Leeuw, E. D., Boevee, A., Hox, J.
- Sensitive Topics in PC and Mobile Web Surveys; 2013; Mavletova, A. M., Couper, M. P.
- Sampling online communities: using triplets as basis for a (semi-) automated hyperlink web crawler.; 2013; Veny, Y.
- Propensity Score Weighting – Can Personality Adjust for Selectivity?; 2013; Glantz, A., Greszki, R.
- GESIS Online Panel Pilot: Results from a Probability-Based Online Access Panel; 2013; Kaczmirek, L., Bandilla, W., Schaurer, I., Struminskaya, B., Weyandt, K.
- Innovation in Data Collection: the Responsive Design Approach; 2013; Bianchi, A., Biffignandi, S.
- Break-off and attrition in the GIP amongst technologically experienced and inexperienced participants...; 2013; Blom, A. G., Bossert, D., Clark, V., Funke, F., Gebhard, F., Holthausen, A., Krieger, U., Wachenfeld...
- Nonresponse and Nonresponse Bias in a Probability-Based Internet Panel; 2013; Blom, A. G., Bossert, D., Funke, F., Gebhard, F., Holthausen, A., Krieger, U.
- Rewards - Money for Nothing?; 2013; Cape, P. J., Martin, P.
- Effects of incentive reduction after a series of higher incentive waves in a probability-based online...; 2013; Struminskaya, B., Kaczmirek, L., Schaurer, I., Bandilla, W.
- Timing of Nonparticipation in an Online Panel: The effect of incentive strategies; 2013; Douhou, S., Scherpenzeel, A.
- Measurement effects in mixed-mode panel surveys; 2013; Lugtig, P. J.
- Experiences from a probability-based Internet panel: Sample, recruitment and participation; 2013; Scherpenzeel, A.
- Participation and engagement in web surveys of the general population: An overview of challenges and...; 2013; Roberts, C.
- How Do Lotteries and Study Results Influence Response Behavior in Online Panels?; 2013; Goeritz, A., Luthe, S. C.
- Sample composition discrepancies in different stages of a probability-based online panel; 2013; Bosnjak, M., Haas, I., Galesic, M., Kaczmirek, L., Bandilla, W., Couper, M. P.
- Sensitive topics in PC Web and mobile web surveys: Is there a difference?; 2012; Mavletova, A. M., Couper, M. P.
- Selection bias of internet panel surveys: A comparison with a paper-based survey and national governmental...; 2012; Tsuboi, S. et al.
- Screenwise panel: Frequently Asked Questions; 2012
- Research company spotlight - Mobile surveys; 2012
- NBCU enlists Google, ComScore to track multiscreen Olympics viewing; 2012; Spangler, T.
- More dirty little secrets of online panel research.; 2012
- Google et Médiamétrie créent une audience bimédia; 2012; Gonzales, P.
- Especially for You: Motivating Respondents in an Internet Panel by Offering Tailored Questions; 2012; Oudejans, M.
- The war against unengaged online respondents; 2012; Gittelman, S. H., Trimarchi, E.
- By the Numbers: Lessons for using online panels in B2B research; 2012; Elsner, N.
- Recruiting in an Internet panel using respondent driven sampling; 2012; Schonlau, M.
- Multi-Language Multi-Continent B2B Community Panel: How B2B research can effectively span the world; 2012; Morden, M., Accomando, E.
- WebSM Study: Survey software features overview ; 2012; Vehovar, V.; Cehovin, G.; Kavcic, L.; Lenar, J.
- Web Panels; 2012; Bethlehem, J., Biffignandi, S.
- The Problem of Self-Selection; 2012; Bethlehem, J.,Biffignandi, S.
- Does survey experience affect respondents’ reported level of satisfaction?; 2012; Schultz Christensen, A., Ladenburg, J.
- Evaluation of an online (opt-in) panel for public participation geographic information systems surveys...; 2012; Brown, G., Weber, D., Zanon, D., de Bie, K.
- Panel Conditioning in Online Survey Panels: Problems of Increased Sophistication and Decreased Engagemeent...; 2012; Adams, A. N., Atkeson, L. R., Karp, J. A.
- Surveying Rare Populations Using a Probabilitybased Online Panel; 2012; Peugh, J., Wright, G.
- Recruiting A Probability Sample For An Online Panel: Effects Of Contact Mode, Incentives, And Information...; 2012; Scherpenzeel, A., Toepoel, V.
- Innovation der Online-Datenerhebung für wissenschaftliche Forschungen. Das niederländische MESS-Projekt...; 2012; Das, M.
- “I think I know what you did last summer” Improving data quality in panel surveys; 2012; Lugtig, P. J.
- Understanding Society Innovation Panel Wave 4: Results from Methodological Experiments; 2012; Burton, J., Budd, S., Gilbert, E., Jaeckle, A., Kaminska, O., Uhrig, S.C. N., Brown, M., Calderwood,...
- The Propensity of Older Respondents to Participate in a General Purpose Survey; 2012; Lynn, P.
- Online Data Collection in the Agro-Food Sector; 2012; Biffignandi, S., Artaz, R.
- Time use data collection using Smartphones: Results of a pilot study among experienced and inexperienced...; 2012; Scherpenzeel, A., Sonck, N., Fernee, H.
- Effect of different stimulus on data quality in online panels; 2012; Zagar, S., Lozar Manfreda, K.
- GESIS Online Access Panel Pilot Study: Recruitment and Panel Maintenance; 2012; Kaczmirek, L., Bandilla, W., Schaurer, I., Struminskaya, B., Weyandt, K.