Web Survey Bibliography
To measure what determines people’s attitudes, definitions or decisions, social science surveys increasingly use vignettes. A vignette typically describes a situation or object as having various attributes (dimensions). Respondents are asked to form a judgment of the hypothetical situation or object described by the vignette. The values (levels) of dimensions are experimentally varied, so that their impact on respondents’ judgments can be estimated. Previous studies on related methods (conjoint analyses and choice experiments) have shown that the order in which dimensions are presented in object descriptions can affect respondents’ judgments. Similarly, it is well known that the order of single item questions, or response options, can substantially affect responses. The underlying mechanisms causing order effects – in vignettes or single item questions – are however not well understood. Drawing on the literature in cognitive psychology and survey methodology, we examine two main research questions: Does the order in which dimensions are presented impact the vignette evaluations and change substantive conclusions? Under which conditions are order effects mostly likely to occur? Using data from a factorial web survey of 300 students we analyze several possible moderators: features of the vignette design (number of dimensions, complexity of the evaluation task, sequence of vignettes, dimension importance), characteristics of respondents (knowledge of the topic, strength of attitudes, cognitive ability), and interactions between these features. Our results show that strong order effects can occur, but only when the vignettes are of a minimum complexity.
Conference Homepage (abstract)
Web Survey Bibliography - 2nd International Conference of the German Priority Programme on Survey Methodology (PPSM) (7)
- The German Access Panel and the Impact of Response Propensities; 2011; Amarov, B., Enderle, T., Muennich, R., Rendtel, U., Zins, S.
- Long-Term Effects in the PPSM Access Panel; 2011; Bartsch, S., Vehre, H., Engel, U.
- Online-Telephone Mixed-Mode Surveys Question Wording Experiments in The Netherlands and Germany; 2011; de Leeuw, E. D., Hox, J.
- First Equals Most Important? Order Effects in Vignette- Based Measurement; 2011; Auspurg, K., Jaeckle, A.
- Equivalence of Measures of Xenophobia in Cross National Survey Research: Findings Based on a Web Probing...; 2011; Bandilla, W., Behr, D., Braun, M., Kaczmirek, L., Majer, S.
- Web Survey Process within the Concept of eSocial Sciences; 2011; Vehovar, V.
- Addressing Panel Attrition in Mobile Phone Panel Surveys: Can Incentives and Switching to Mobile Web...; 2011; Busse, B., Fuchs, M.