Web Survey Bibliography
Survey data are unavoidably contaminated with measurement error. We focus on formatting error, the increase in con dential intervals of survey estimates, making it more difficult to detect existing dierences. Formatting error happens when respondents do not fi nd an option on a rating scale that perfectly reflects their true value. The dierence between the true value and the chosen response option is formatting error.
From a theoretical point of view, continuous visual analogue scales (VASs) have on the individual as well as on the aggregated level an expected formatting error of zero, because there is a perfectly tting option for every graduation of the true value. An empirical determination of formatting error with VASs is pending and it is unclear, if populations with a low formal education are able to use VASs in a meaningful way.
Formatting error with categorical scale is dierent for single individual variables and for aggregate data. In the first case, it depends on the number of categories only. In the second case it is additionally in uenced by the actual distribution of values in the sample. We simulated differently distributed data (e.g. from uniform distributions, narrow and wide normal distributions, chi-square and exponential distributions) to determine the expected formatting error with categorical scales consisting of 3 to 21 categories. Overall (N = 1909), we found a very low mean empirical formatting error of M = -1.24 percentage points (SD = 3.05). Ratings on plain VASs without any marker (n = 167) were worse (M = -2.48, SD = 2.64) and formal education (below college: M = -2.87, SD = 2.93; at least college: M = -1.71, SD = 1.61) made a statistically signi cant dierence: F(1, 166) = 7.56, p < .01, eta2 = .04. VASs with ten markers (n = 181) lead to the smallest formatting error (M = -0.41, SD = 1.55) for respondents with a low education (M = -0.47, SD = 1.63) and respondents with a high formal education (M = -0.30, SD = 1.39), F(1, 180) < 1. For individual variables, the empirical formatting error for VASs with ten markers is even with respondents with a low formal education lower than the expected formatting error for categorical scales up to 50 options. Overall, the authors strongly recommend considering VASs in computer-based self-administered questionnaires for the sake of more con dent survey estimates.
Conference homepage(abstract)
Web Survey Bibliography - Reips, U. -D. (90)
- Comparison of psychometric properties of internet versions of the Marlowe-Crowne Social Desirability...; 2013; Vesteinsdottir, V., Reips, U. -D., Joinson, A. N., Porsdottir, F.
- True experimental data collection on the Internet; 2013; Reips, U. -D., Krantz, J. H.
- Askito: An open source Web questionnaire tool; 2013; Reips, U. -D., Heilmann, T.
- Studying Migrants with the Help of the Internet: Methods from Psychology; 2012; Reips, U. -D., Buffardi, L.
- Psychometric properties of an internet administered version of the Marlowe-Crowne Social Desirability...; 2012; Vesteinsdottir, V., Reips, U. -D., Joinson, A. N., Porsdottir, F.
- WEBDATANET: A Network on Web-based Data Collection, Methodological Challenges, Solutions and Implementation...; 2012; Steinmetz, S., Kaczmirek, L., de Pedraza, P., Reips, U. -D., Tijdens, K., Lozar Manfreda, K., et, al...
- Better low-tech than sorry: How technophile questionnaires may affect psychological representativeness...; 2012; Funke, F., Reips, U. -D.
- Why semantic differentials in Web-based research should be made from visual analogue scales and not...; 2012; Funke, F., Reips, U. -D.
- Using the Internet to collect data; 2012; Reips, U. -D.
- Using Amazon's Mechanical Turk for the recruitment of participants in Internet-based research; 2011; Reips, U. -D., Buffardi, L., Kuhlmann, T.
- Slider Scales Causing Serious Problems With Less Educated Respondents; 2011; Funke, F., Reips, U. -D., Thomas, R. K.
- Dropout in Web-based studies: Methodology; 2011; Reips, U. -D.
- Sliders for the Smart: Type of Rating Scale on the Web Interacts With Educational Level; 2011; Funke, F., Reips, U. -D., Thomas, R. K.
- Advice in Surveying the General Public Over the Internet; 2010; Dillman, D. A., Reips, U. -D., Matzat, U.
- Design and formatting in Internet-based research; 2010; Reips, U. -D.
- What are participants doing while filling in an online questionnaire: A paradata collection tool and...; 2010; Stieger, S., Reips, U. -D.
- Internet experiments: methods, guidelines, metadata; 2009; Reips, U. -D.
- Twisting rating scales in Web surveys: Slider scales versus categorical scales of horizontal versus...; 2009; Funke, F. Reips, U. -D. Thomas, R. K.
- Semantic differentials made from visual analogue scales: Expanding the survey designer's menu; 2009; Funke, F., Reips, U. -D.
- Making small effects observable: Reducing error by using visual analogue scales; 2009; Funke, F., Reips, U. -D.
- Increasing Confidence in Survey Estimates with Visual Analogue Scales; 2009; Funke, F., Reips, U. -D., Thomas, R. K.
- Twisting Rating Scales: Horizontal versus Vertical Visual Analogue Scales versus Categorical Scales...; 2009; Funke, F., Reips, U. -D.
- Yes, VASs can! Increasing the accuracy of survey measurements with computerized visual analogue scales...; 2009; Funke, F., Reips, U. -D.
- True Web experiments; 2009; Reips, U. -D.
- Results from 6 Independent Web Experiments Comparing Visual Analogue Scales with Categorical Scales; 2009; Funke, F., Reips, U. -D.
- Does the seriousness check really sieve out datasets with bad data quality?; 2009; Stieger, S., Reips, U. -D.
- Investigating the High Hurdle Technique; 2009; Frauendorfer, D., Reips, U. -D.
- The impact of privacy concerns on data collection; 2009; Reips, U. -D., Joinson, A. N., Buchanan, T., Schofield Paine, C.
- GPCP: A German Version of the Scale for Online Privacy Concern and Protection for Use on the Internet...; 2009; Oostlander, J., Reips, U. -D., Buchanan, T.
- Formatting Error with Visual Analogue Scales in Web Surveys; 2009; Funke, F., Reips, U. -D.
- Internet questionnaires in e-health contexts: Non-response to sensitive items; 2009; Reips, U. -D., Buchanan, T., Joinson, A. N., Paine, C.
- Reaction times in Internet-based versus laboratory research: Potential problems and a solution; 2009; Reips, U. -D.
- Collecting data in surfer's paradise: Internet-mediated research yesterday, now, and tomorrow; 2009; Reips, U. -D.
- A decade of Internet-based data collection: Time is ripe for combining e-learning with i-science; 2009; Reips, U. -D.
- Experimentation within surveys; 2009; Reips, U. -D.
- Investigating causal relationships with power: Online experiments; 2009; Reips, U. -D.
- Internet-basierte Messung sozialer Erwünschtheit: Theoretische Grundlagen und experimentelle Untersuchung...; 2008; Kaufmann, E., Reips, U. -D.
- Assessing Semantic Differentials with Visual Analogue Scales in Web Surveys; 2008; Funke, F., Reips, U. -D.
- Visual Analogue Scales in Cross Cultural Web Surveys ; 2008; Funke, F., Reips, U. -D., Thomas, R. K.
- Interval-level measurement with visual analogue scales in Internet-based research: VAS generator; 2008; Reips, U. -D., Funke, F.,
- Visual Analogue Scales Versus Categorical Scales: Respondent Burden, Cognitive Depth, and Data Quality...; 2008; Funke, F.,Reips, U. -D.
- Measuring self-disclosure online: Blurring and non-response to sensitive items in web-based surveys; 2008; Joinson, A. N., Paine, C., Buchanan, T., Reips, U. -D.
- Privacy, Trust and Self-Disclosure to Web-Based Surveys; 2007; Joinson, A. N., Paine, C., Buchanan, T., Reips, U. -D.
- Response time measurement in the lab and on the Web: A comparison; 2007; Galesic, M., Reips, U. -D., Kaczmirek, L., Czienskowski, U., Liske, N., von Oertzen , T.
- Improving Data Quality in Web Surveys with Visual Analogue Scales; 2007; Funke, F., Reips, U. -D.
- Experiments on non- response in internet- based research; 2007; Reips, U. -D.
- Improving Data Quality in Web Surveys with Visual Analogue Scales; 2007; Funke, F., Reips, U. -D.
- VASgenerator.net - A Web-Based Tool for Creating Visual Analogue Scales ; 2007; Reips, U. -D., Funke, F.
- Dynamic Forms: Online Surveys 2.0 ; 2007; Funke, F., Reips, U. -D.
- Forced response in online surveys: bias from reactance and an increase in sex-specific dropout; 2007; Stieger, S., Reips, U. -D., Voracek, M.
