Web Survey Bibliography
(a) Relevance & Research Question:
Social scientists are often reluctant to rely on data from online access panels or other web surveys because they fear that the general data quality may be seriously flawed and the results are thus neither valid nor reliable . The third major concern of representative sampling is not always important, e.g. in case of experimental designs or studies that are only used for analyzing relationships as opposed to comparing distributions and mean values. Our research question is, whether speeders in online surveys jeopardize results due to a lack of validity and reliability of results.
(b) Methods & Data:
In a research project on basic human values, fielded in 2010, we have implemented an online questionnaire and gathered data using the ‘Unipark’ online access panel. The 20 minute instrument includes among other measurements the 40 item Schwartz Portrait Value Questionnaire (PVQ) over five consecutive pages. We use confirmatory factor analyses (CFA) and structural equation models (SEM) to analyze our data.
(c) Results:
When analyzing the data we hardly faced any item non-response but have discovered many comparatively fast respondents using different speed thresholds. In order to cross-check our results we compared the results of CFA and SEM from two separate surveys (each: n > 750) using independent samples and we compare models from respondents at different speeds. We found a high degree of homogeneity between the two different samples and the models of slow respondents and speeders while controlling for straightliners on our Likert scale.
(d) Added Value:
The results indicate that value research may profit from online access panels both to pretest complex instruments and to validate results, e.g. by applying structural equation models. Although the quality of representative samples is not yet given for most online panels, the tentative results of our fundamental research indicates that speeders do not necessarily jeopardize the quality of the data and thus the validity and the reliability of results. This is an important insight for those social scientists usually relying on cost-intensive traditional PAPI and CATI data collection.
Conference Homepage (abstract) / (presentation)
Web survey bibliography - General Online Research Conference (GOR) 2011 (17)
- Sampling v. Scale: An investigation the tension between convenience sampling, response rates, probability...; 2011; Garland, P.
- Effectiveness and consequences of various recruitment methods in psychological research: case study; 2011; Poltorak, M.
- A new approach to the analysis of survey drop-out. Results from Follow-up Surveys in the German Longitudinal...; 2011; Rossmann, J., Blumenstiel, J. E., Steinbrecher, M.
- Tracking the decision-making process – Findings from an Online Rolling Cross-Section Panel Study...; 2011; Faas, T.
- Should we use the progress bar in online surveys? A meta-analysis of experiments manipulating progress...; 2011; Callegaro, M., Yang, Y., Villar, A.
- From "Web Questions" to "Propensity Score Weighting": An Evaluation of Topics and...; 2011; Welker, M., Taddicken, M.
- Rich Profiles – Or: What's the problem with self-disclosure data?; 2011; Tress, F.
- Who are leaving our panel: panel attrition and personality traits; 2011; Marchand, M.
- Mobile Research Apps – Adding New Capabilities to Market Research; 2011; Rieber, D.
- The influence of personality traits and motives for joining on participation behavior in online panels...; 2011; Keusch, F.
- Asking sensitive questions in a recruitment interview for an online panel: the income question; 2011; Schaurer, I., Struminskaya, B., Kaczmirek, L., Bandilla, W.
- Speeders in Online Value Research: Cross-checking results of fast and slow respondents in two separate...; 2011; Beckers, T., Siegers, P., Kuntz, A.
- Effects of survey question clarity on data quality; 2011; Lenzner, T.
- Respondent Characteristics as Explanations for Uninformative Survey Response: Sources of Nondifferentiation...; 2011; Van Meurs, L., Klausch, L. T., Schoenbach, K.
- Response Quantity, Response Quality, and Costs of Building an Online Panel via Social Contacts.; 2011; Toepoel, V.
- The Influence Of The Direction Of Likert-Type Scales In Web Surveys On Response Behavior In Different...; 2011; Keusch, F.
- Social desirability and self-reported health risk behaviors in web-based research: three longitudinal...; 2010; Crutzen, R., Goeritz, A.