Web Survey Bibliography
Relevance & Research Question: Self-administered online surveys put respondents into an essentially anonymous and uncontrolled response situation. This raises worries on potentially biased or uninformative answers, such as nondifferentiation – always using the same score on all items offered – which may harm the measurement accuracy of population statistics. Our presentation explores the question which respondents are inclined to give such answers.
Methods & Data: For our study, longitudinal observations from a large commercial online survey panel in The Netherlands were available: the Appreciation Panel (fieldwork by Intomart GfK on behalf of NPO, the Dutch Public Broadcasting Organisation. Nondifferentiation behavior was identified in every single survey of the panel for a time frame of six months in 2009 (totaling to 502,750 completed online questionnaires). In this way a history of panel (nondifferentiation) behavior was created for each of over 7,700 active panel members. Subsequently a cross-sectional online survey was designed to survey possible determinants of response behavior. The survey was conducted post-hoc with a stratified probability sample of 1,200 respondents.
Results: Analyses based on data from a large-scale online panel indicate that not only respondents’ perception of effort caused by a survey explains their behavior. Also more abstract social behavioral norms, individual moral obligations and the norm of ‘honest behavior’ are related to nondifferentiation behavior. However, extrinsic motivation to participate in the panel because of a monetary incentive is found unrelated. These results imply that survey researchers have somewhat limited ways to reduce the effects of factors causing uninformative behaviors. Using monetary incentives to encourage panel participation is not harmful to the quality of answers, but it is recommended to limit respondents’ perception of effort.
Added Value: Very few examples have been published about nondifferentiation in applied online market research. The method presented offers an example of applied research what respondents are inclined to give nondifferentiated responses and how nondifferentiation in combination with other indicators such as response time is used to identify low quality responses in online research.
Conference Homepage (abstract) / (presentation)
Web survey bibliography - Klausch, L. T. (18)
- Mixed Mode Research: Issues in Design and Analysis; 2016; Hox, J.; De Leeuw, E. D.; Klausch, L. T.
- A look into the challenges of mixed-mode surveys; 2016; Klausch, L. T.
- Tracking the Representativeness of an Online Panel Over Time ; 2016; Klausch, L. T.; Scherpenzeel, A.
- Selection error in single- and mixed mode surveys of the Dutch general population; 2015; Hox, J., Klausch, L. T., Schouten, B.
- Evaluating mixed-mode redesign strategies against benchmark surveys: the case of the Crime Victimization...; 2014; Klausch, L. T., Hox, J., Schouten, B.
- The use of within-subject experiments for estimating measurement effects in mixed-mode surveys ; 2014; Klausch, L. T., Schouten, B., Hox, J.
- Pret met panels [Fun online]; 2013; Roberts, A., de Leeuw, E. D., Hox, J., Klausch, L. T., de Jongh, A.
- Leuker kunnen wij het wel maken. Online vragenlijst design: standaard matrix of scrollmatrix (We can...; 2013; Roberts, A., de Leeuw, E. D., Hox, J., Klausch, L. T., de Jongh, A.
- Measurement Effects of Survey Mode on the Equivalence of Attitudinal Rating Scale Questions; 2013; Klausch, L. T., Hox, J., Hox, J., Schouten, B.
- Estimating Measurement Effects of Survey Modes From Between and Within Subject Designs; 2013; Klausch, L. T., Hox, J., Schouten, B.
- Design of Web Questionnaires: Matrix Questions or Single Question Formats ; 2012; de Leeuw, E. D., Hox, J., Klausch, L. T., Roberts, A., de Jongh, A.
- Disentangling Mode-Specific Selection and Measurement Bias in Social Surveys; 2012; Buelens, B., van der Laan, J., Schouten, B., Klausch, L. T., van der Brakel, J., Burger, J.
- Assessing Measurement Equivalence and Bias of Questions in Mixed-mode Surveys Under Controlled Sample...; 2012; Klausch, L. T., Hox, J., Schouten, B.
- The Representativity of Web Surveys of the General Population compared to Traditional Modes and Mixed...; 2012; Klausch, L. T., Schouten, B., Hox, J.
- Matrix vs. Single Question Formats in Web Surveys: Results from a large scale experiment; 2012; Klausch, L. T., de Leeuw, E. D., Hox, J., de Jongh, A., Roberts , A.
- Disentangling relative mode effects for the web survey mode in the Safety Monitor; 2011; Schouten, B., van de Brakel, J., Buelens, B., Klausch, L. T., van der Laan, J.
- Respondent Characteristics as Explanations for Uninformative Survey Response: Sources of Nondifferentiation...; 2011; Van Meurs, L., Klausch, L. T., Schoenbach, K.
- Testing between-mode measurement invariance under controlled selectivity conditions; 2011; Klausch, L. T.