Web Survey Bibliography
Analysts working with data generated by different modes of data collection often want to be sure that their measurements are comparable. If a set of questions is designed to measure the same latent trait, confirmatory factor analysis (CFA) is a useful analytic tool for this purpose. It can be applied to assess, whether properties, such as measurement error, the association between latent traits and questions (measurement invariance) and the means of latent traits, are equivalent across survey modes. We illustrate an application using empirical data from an experiment based on a national probability sample, in which 4048 respondents were randomly assigned to either face-to-face, telephone, mail or web interviewing. Two related traits were measured with three questions respectively, “moral support of the police” and the “obligation to obey”, which form the basis of our CFA model. The association between latent traits and questions was invariant across modes. However, measurement errors differed between modes. In particular, the self-administered modes yielded more reliable indicators than the interviewer modes. Moreover, we find systematic bias between modes on the mean of one of the traits. The effect signs suggest that respondents gave socially desirable answers in the intervieweradministered modes. A particularity of survey modes is that sample compositions are often heterogeneous. If the selective process is correlated with model elements, such as traits, it can bias invariance tests and decrease fit. We illustrate available options to adjust for this problem, for example propensity score methods or covariate adjustment, all based on the use of auxiliary variables, such as socio-demographics. In conclusion, self-administered modes seem to produce measurements of lower quality than interviewer-administered modes with respect to random error and systematic bias. Modes may thus affect both, the error variance and bias of an estimate. An effect can be suspected particularly between interviewer and non-interviewer modes.
Conference Homepage (abstract)
Web survey bibliography - Klausch, L. T. (18)
- Mixed Mode Research: Issues in Design and Analysis; 2016; Hox, J.; De Leeuw, E. D.; Klausch, L. T.
- A look into the challenges of mixed-mode surveys; 2016; Klausch, L. T.
- Tracking the Representativeness of an Online Panel Over Time ; 2016; Klausch, L. T.; Scherpenzeel, A.
- Selection error in single- and mixed mode surveys of the Dutch general population; 2015; Hox, J., Klausch, L. T., Schouten, B.
- Evaluating mixed-mode redesign strategies against benchmark surveys: the case of the Crime Victimization...; 2014; Klausch, L. T., Hox, J., Schouten, B.
- The use of within-subject experiments for estimating measurement effects in mixed-mode surveys ; 2014; Klausch, L. T., Schouten, B., Hox, J.
- Pret met panels [Fun online]; 2013; Roberts, A., de Leeuw, E. D., Hox, J., Klausch, L. T., de Jongh, A.
- Leuker kunnen wij het wel maken. Online vragenlijst design: standaard matrix of scrollmatrix (We can...; 2013; Roberts, A., de Leeuw, E. D., Hox, J., Klausch, L. T., de Jongh, A.
- Measurement Effects of Survey Mode on the Equivalence of Attitudinal Rating Scale Questions; 2013; Klausch, L. T., Hox, J., Hox, J., Schouten, B.
- Estimating Measurement Effects of Survey Modes From Between and Within Subject Designs; 2013; Klausch, L. T., Hox, J., Schouten, B.
- Design of Web Questionnaires: Matrix Questions or Single Question Formats ; 2012; de Leeuw, E. D., Hox, J., Klausch, L. T., Roberts, A., de Jongh, A.
- Disentangling Mode-Specific Selection and Measurement Bias in Social Surveys; 2012; Buelens, B., van der Laan, J., Schouten, B., Klausch, L. T., van der Brakel, J., Burger, J.
- Assessing Measurement Equivalence and Bias of Questions in Mixed-mode Surveys Under Controlled Sample...; 2012; Klausch, L. T., Hox, J., Schouten, B.
- The Representativity of Web Surveys of the General Population compared to Traditional Modes and Mixed...; 2012; Klausch, L. T., Schouten, B., Hox, J.
- Matrix vs. Single Question Formats in Web Surveys: Results from a large scale experiment; 2012; Klausch, L. T., de Leeuw, E. D., Hox, J., de Jongh, A., Roberts , A.
- Disentangling relative mode effects for the web survey mode in the Safety Monitor; 2011; Schouten, B., van de Brakel, J., Buelens, B., Klausch, L. T., van der Laan, J.
- Respondent Characteristics as Explanations for Uninformative Survey Response: Sources of Nondifferentiation...; 2011; Van Meurs, L., Klausch, L. T., Schoenbach, K.
- Testing between-mode measurement invariance under controlled selectivity conditions; 2011; Klausch, L. T.