Web Survey Bibliography
Relevance and Research Question:
It is well-documented that online surveys elicit higher reports of socially undesirable behavior than interviewer-administered surveys. However, there are possible exceptions, where the form of the question may inhibit the revelation of prejudicial attitudes. In research exploring race of interviewer effects, Krysan and Couper (2003) found some instances where white respondents (for example) gave more negative responses to interviewers than to computerized instruments. In qualitative debriefings, some respondents noted that talking to an interviewer gave them an opportunity to explain their choice of responses; in the CASI condition (as on the Web; see Krysan and Couper, 2005), they could only pick one of the response options, without the opportunity to justify their choice. We called this the “I’m not a racist, but…” phenomenon. An online experiment was designed to explore the hypothesis that, when given an opportunity to explain or clarify their answers, respondents will give more prejudicial responses.
Methods and Data:
Two experiments were embedded in the LISS online probability-based panel in the Netherlands. In both cases a set of 9 items on attitudes toward immigrants was asked. In the first study, conducted in August 2009 (n=4639), a random half received an open-ended question on a separate page following each closed question. In the second study, conducted in December 2010 (n=5328), for a random half of respondents, an optional open-ended comment appeared below each closed-ended question on the same page.
The results provide support for the hypothesis. In both cases, respondents given the open question gave significantly more prejudiced responses (F[1, 4352]=25.6, p<.001 for Exp. 1 and F[1, 5326]=7.1, p=0.008 for Exp. 2) than those getting only the closed-ended question. However, contrary to expectation, the effect was larger in experiment 1 than 2. We explore this finding in greater detail, examining both responses to individual items and those who made use of the text box to offer comments.
This study suggests value in giving respondents the opportunity to voice their opinions in their own words, rather than just requiring them to agree or disagree with one of the response options.
GOR Homepage (abstract) / (presentation)
Web Survey Bibliography - Conferences, workshops, tutorials, presentations (3066)
- The challenge of a mixed-mode design survey and new IT tools application: the case of the Italian Structure...; 2013; Cardinaleschi, S., De Santis, S., Rocci, F., Spinelli, V.
- MOTUS: Modular online Time-Use Survey; 2013; Joeri, M., Ignace, G., van Tienoven, T. P., Djiwo, W.
- The Design of the Online Questionnaire of the Italian Population Census ; 2013; Tininini, L., Virgillito, A.
- Experiments in Obtaining Data Linkage Consent in Web Surveys ; 2013; Sakshaug, J. W., Kreuter, F.
- The behaviour of respondents while filling in a web questionnaire: the case of the Italian business...; 2013; Masselli, M., Nuccitelli, A.
- A web-based Census of services: an ISTAT evolutionary study ; 2013; Cesaro, A., Palazzi, B., Paterniti, M., Ranaldi, P.
- A web based management system for addressing census complexity: the Italian experience; 2013; Bruno, M., Giacummo, M., Silipo, M., Vaste, G.
- Response Burden in Official Business Surveys: Measurement and Reduction Practices of National Statistical...; 2013; Giesen, D., Bavdaz, M., Loefgren, T., Raymond-Blaess, V.
- Factors affecting the decision to participate in the internet option for the 2010 Census of Korea ; 2013; Lee, E., Kim, S.
- Internet as a new source of information for the production of official statistics. Experiences of Statistics...; 2013; Heerschap, N.
- Web panel surveys – can they be designed and used in a scientifically sound way?; 2013; Svensson , J.
- A standard with quality indicators for web panel surveys: a Swedish example; 2013; Nyfjaell, M.
- Web Panels for Official Statistics? ; 2013; Bethlehem, J., Cobben, F.
- Assessing Nonresponse Bias in the Green Technologies and Practices Survey; 2013; Meekins, B., Sverchkov, M., Stang, S.
- Using an Item Response Theory Approach to Measure Survey Mode of Administration Effects: Analysis of...; 2013; Mariano, L. T., Elliott, M. N.
- The Role of Mode Preference Questions in Predicting Mode-Specific Response Propensities; 2013; Lynn, P., Kaminska, O.
- How Does Online Survey Mode Affect Answers to Customer Feedback Loyalty Surveys?; 2013; Gupta, A., Lee, J.
- Using Internet Panel Surveys for Behavioral Health Surveillance; 2013; Gotway Crawford, C., Okoro, C. A., Dhingra, S., Akcin, H., Zhao, G., Ford, D., Pierannunzi, C.
- Model-Based Mode of Data Collection Switching from Internet to Mail in the American Community Survey; 2013; Chesnut, J.
- Estimating Mode Effects Without Bias: A Randomized Experiment to Compare Mode Effects Between Face-to...; 2013; Rivers, D., Vavreck, L.
- How Mobile Stacks Up to Traditional Online: A Comparison of Studies; 2013; Knowles, R.
- Respondent Rewards: Money for Nothing?; 2013; Martin, P.
- Did I Do That? How Trap Questions Can Hurt Data Quality; 2013; Phillips, K.
- Web Coverage in the UK and its Potential Impact on General Population Web Surveys; 2013; Callegaro, M.
- Issues of Coverage and Sampling in Web Surveys for the General Population; 2013; Lynn, P.
- Gamification Master Class; 2013; Puleston, J.
- Research Communities in Asia Pacific: A review of similarities and contrasts; 2013; Poynter, R.
- Measuring Up: Impact of mobile and segmentation on respondent behaviour; 2013; Luck, K.
- Best of Both Worlds? Can we make convenience samples representative?; 2013; Doe, P.
- Multimode, Global Scale Usage: Understanding respondent scale usage across borders and devices; 2013; Pettit, F. A., Courtright, M.
- Why Big Data is a Small Idea…and Why You Shouldn’t Worry So Much; 2013; Needel, S.
- Mode effect analysis and adjustment in a split-sample mixed-mode Web/CATI survey; 2013; Kolenikov, S., Kennedy, C.
- Exploring factors associated with respondent mode choice for surveys using mobile devices.; 2013; Walton, L.
- Responsive design for mixed-mode panel data; 2013; Bianchi, A., Biffignandi, S.
- Adjusting for bias in a mixed-mode CAWI survey on University students ; 2013; Clerici, R., Giraldo, A.
- Comparative analysis of data from web and face-to-face surveys. A case study on e-commerce in young...; 2013; Cappello, C., Pellegrino, D.
- Insights into Action Profiling shopping occasions for retailers through mobile and online research; 2013; Churkina, O., Morris, T.
- Mobilizing your Branded Panel: Panel data quality during the smartphone transition; 2013; Kugel, C., Brien, D., Blechman, J.
- Evaluating the left‐right dimension: Category Selection Probing conducted in an online access...; 2013; Huefken , V.
- From Face‐to‐face to Web: Consequences for Measurement of Complex and Open‐ended Questions...; 2013; Villar, A., Fitzgerald, R., Martin, P., Harrison, E., Gatrrell, L.
- Computers, Tablet & Smart Phones: The Truth about Web‐based Surveys; 2013; Merle, P., Gearhart, S., Craig, C., Rahimi, M., Brooks, M. E., Vandyke, M.
- Explaining Interview Duration in Web Surveys on Political Attitudes and Behavior: A Multilevel Approach...; 2013; Gummer, T., Rossmann, J.
- An Examination of the Relationship Between Pretest Method Results and Data Quality; 2013; Maitland, A.
- Associations Between Interactional Indicators of Problematic Questions and Systems for Coding Question...; 2013; Dykema, J., Schaeffer, N. C., Garbarski, D.
- The Technology Behind Mobile Research: Browsers vs. Apps ; 2013; Macer, T.
- Methodological, legal and technical perspectives on the feasibility of web survey paradata in German...; 2013; Sattelberger, S.
- Specific mixed-mode methodologies to include sensory disabled people in quantitative surveys; 2013; Sebastien, F., Marc, J., Patrick, I.
- An imputation approach for analyzing mixed-mode surveys; 2013; Kim, J. K., Park, S., Kim, S. Y.
- Investigating the Bias of Alternative Statistical Inference Methods in Sequential Mixed-Mode Surveys; 2013; Suzer-Gurtekin, Z., Heeringa, S. G., Valliant, R. L.
- Random versus Systematic Error in a Mixed Mode Online-Telephone Survey; 2013; Hox, J., Scherpenzeel, A., Boeve, A., Boeve, A., de Leeuw, E. D.