Web Survey Bibliography
Compared to laboratory studies, participants much more likely drop out in Web studies, particularly if sensitive questions are asked. The high hurdle technique is thought to control drop out by maximizing de-motivating factors at the very beginning of a study or even earlier to reduce subsequent drop out, but previous research is inconclusive as to its impact and process (Göritz & Stieger, 2007; Reips, 2002). To investigate the technique’s dependence on other factors and its impact on drop out behavior, we conducted a Web experiment in which we (1) measured pre-experimental intended seriousness and (2) manipulated the high hurdle and the placement of sensitive items. Twelve different conditions resulted from the 2 x(seriousness) x 2 (high hurdle) x 3 (placement of sensitive items) between-subjects design.
Dependent measures were drop out, data quality and reaction time. After the seriousness check item, 396 Students answered a questionnaire about sport and nutrition. The high hurdle was placed in the introduction and was created by saying that the questionnaire contains sensitive items, which might make the person feel uncomfortable. In the control condition, there was no such high hurdle. The sensitive items were set either at the beginning, in the middle, or at the end of the experiment.
There was a significantly larger drop out directly after the high hurdle introduction, compared to the control condition (p<.001), i.e. the high hurdle served the first of its purposes. Also, there was an interaction with intended seriousness.
Further analyses showed that satisficer (people who secede to deliver accurate data in favor of roundings or estimates, likely due to decreasing motivation) had more missings than nonsatisficers, in contradiction to widespread intuitions. The assumption that participants take a longer time to answer sensitive items was confirmed (p=.005). Also, at sensitive items the frequency of chosen „don’t-want-to-answer“ options exceeded the amount of missings, whereas at non-sensitive items more missings than „don’t-want-to-answer“ options were counted. Implications of these results for research and questionnaire design will be discussed in the presentation.
Conference homepage (abstract)
Web Survey Bibliography - Reips, U. -D. (90)
- Comparison of psychometric properties of internet versions of the Marlowe-Crowne Social Desirability...; 2013; Vesteinsdottir, V., Reips, U. -D., Joinson, A. N., Porsdottir, F.
- True experimental data collection on the Internet; 2013; Reips, U. -D., Krantz, J. H.
- Askito: An open source Web questionnaire tool; 2013; Reips, U. -D., Heilmann, T.
- Studying Migrants with the Help of the Internet: Methods from Psychology; 2012; Reips, U. -D., Buffardi, L.
- Psychometric properties of an internet administered version of the Marlowe-Crowne Social Desirability...; 2012; Vesteinsdottir, V., Reips, U. -D., Joinson, A. N., Porsdottir, F.
- WEBDATANET: A Network on Web-based Data Collection, Methodological Challenges, Solutions and Implementation...; 2012; Steinmetz, S., Kaczmirek, L., de Pedraza, P., Reips, U. -D., Tijdens, K., Lozar Manfreda, K., et, al...
- Better low-tech than sorry: How technophile questionnaires may affect psychological representativeness...; 2012; Funke, F., Reips, U. -D.
- Why semantic differentials in Web-based research should be made from visual analogue scales and not...; 2012; Funke, F., Reips, U. -D.
- Using the Internet to collect data; 2012; Reips, U. -D.
- Using Amazon's Mechanical Turk for the recruitment of participants in Internet-based research; 2011; Reips, U. -D., Buffardi, L., Kuhlmann, T.
- Slider Scales Causing Serious Problems With Less Educated Respondents; 2011; Funke, F., Reips, U. -D., Thomas, R. K.
- Dropout in Web-based studies: Methodology; 2011; Reips, U. -D.
- Sliders for the Smart: Type of Rating Scale on the Web Interacts With Educational Level; 2011; Funke, F., Reips, U. -D., Thomas, R. K.
- Advice in Surveying the General Public Over the Internet; 2010; Dillman, D. A., Reips, U. -D., Matzat, U.
- Design and formatting in Internet-based research; 2010; Reips, U. -D.
- What are participants doing while filling in an online questionnaire: A paradata collection tool and...; 2010; Stieger, S., Reips, U. -D.
- Internet experiments: methods, guidelines, metadata; 2009; Reips, U. -D.
- Twisting rating scales in Web surveys: Slider scales versus categorical scales of horizontal versus...; 2009; Funke, F. Reips, U. -D. Thomas, R. K.
- Semantic differentials made from visual analogue scales: Expanding the survey designer's menu; 2009; Funke, F., Reips, U. -D.
- Making small effects observable: Reducing error by using visual analogue scales; 2009; Funke, F., Reips, U. -D.
- Increasing Confidence in Survey Estimates with Visual Analogue Scales; 2009; Funke, F., Reips, U. -D., Thomas, R. K.
- Twisting Rating Scales: Horizontal versus Vertical Visual Analogue Scales versus Categorical Scales...; 2009; Funke, F., Reips, U. -D.
- Yes, VASs can! Increasing the accuracy of survey measurements with computerized visual analogue scales...; 2009; Funke, F., Reips, U. -D.
- True Web experiments; 2009; Reips, U. -D.
- Results from 6 Independent Web Experiments Comparing Visual Analogue Scales with Categorical Scales; 2009; Funke, F., Reips, U. -D.
- Does the seriousness check really sieve out datasets with bad data quality?; 2009; Stieger, S., Reips, U. -D.
- Investigating the High Hurdle Technique; 2009; Frauendorfer, D., Reips, U. -D.
- The impact of privacy concerns on data collection; 2009; Reips, U. -D., Joinson, A. N., Buchanan, T., Schofield Paine, C.
- GPCP: A German Version of the Scale for Online Privacy Concern and Protection for Use on the Internet...; 2009; Oostlander, J., Reips, U. -D., Buchanan, T.
- Formatting Error with Visual Analogue Scales in Web Surveys; 2009; Funke, F., Reips, U. -D.
- Internet questionnaires in e-health contexts: Non-response to sensitive items; 2009; Reips, U. -D., Buchanan, T., Joinson, A. N., Paine, C.
- Reaction times in Internet-based versus laboratory research: Potential problems and a solution; 2009; Reips, U. -D.
- Collecting data in surfer's paradise: Internet-mediated research yesterday, now, and tomorrow; 2009; Reips, U. -D.
- A decade of Internet-based data collection: Time is ripe for combining e-learning with i-science; 2009; Reips, U. -D.
- Experimentation within surveys; 2009; Reips, U. -D.
- Investigating causal relationships with power: Online experiments; 2009; Reips, U. -D.
- Internet-basierte Messung sozialer Erwünschtheit: Theoretische Grundlagen und experimentelle Untersuchung...; 2008; Kaufmann, E., Reips, U. -D.
- Assessing Semantic Differentials with Visual Analogue Scales in Web Surveys; 2008; Funke, F., Reips, U. -D.
- Visual Analogue Scales in Cross Cultural Web Surveys ; 2008; Funke, F., Reips, U. -D., Thomas, R. K.
- Interval-level measurement with visual analogue scales in Internet-based research: VAS generator; 2008; Reips, U. -D., Funke, F.,
- Visual Analogue Scales Versus Categorical Scales: Respondent Burden, Cognitive Depth, and Data Quality...; 2008; Funke, F.,Reips, U. -D.
- Measuring self-disclosure online: Blurring and non-response to sensitive items in web-based surveys; 2008; Joinson, A. N., Paine, C., Buchanan, T., Reips, U. -D.
- Privacy, Trust and Self-Disclosure to Web-Based Surveys; 2007; Joinson, A. N., Paine, C., Buchanan, T., Reips, U. -D.
- Response time measurement in the lab and on the Web: A comparison; 2007; Galesic, M., Reips, U. -D., Kaczmirek, L., Czienskowski, U., Liske, N., von Oertzen , T.
- Improving Data Quality in Web Surveys with Visual Analogue Scales; 2007; Funke, F., Reips, U. -D.
- Experiments on non- response in internet- based research; 2007; Reips, U. -D.
- Improving Data Quality in Web Surveys with Visual Analogue Scales; 2007; Funke, F., Reips, U. -D.
- VASgenerator.net - A Web-Based Tool for Creating Visual Analogue Scales ; 2007; Reips, U. -D., Funke, F.
- Dynamic Forms: Online Surveys 2.0 ; 2007; Funke, F., Reips, U. -D.
- Forced response in online surveys: bias from reactance and an increase in sex-specific dropout; 2007; Stieger, S., Reips, U. -D., Voracek, M.
