Web Survey Bibliography
Analysts working with data generated by different modes of data collection often want to be sure that their measurements are comparable. If a set of questions is designed to measure the same latent trait, confirmatory factor analysis (CFA) is a useful analytic tool for this purpose. It can be applied to assess, whether properties, such as measurement error, the association between latent traits and questions (measurement invariance) and the means of latent traits, are equivalent across survey modes. We illustrate an application using empirical data from an experiment based on a national probability sample, in which 4048 respondents were randomly assigned to either face-to-face, telephone, mail or web interviewing. Two related traits were measured with three questions respectively, “moral support of the police” and the “obligation to obey”, which form the basis of our CFA model. The association between latent traits and questions was invariant across modes. However, measurement errors differed between modes. In particular, the self-administered modes yielded more reliable indicators than the interviewer modes. Moreover, we find systematic bias between modes on the mean of one of the traits. The effect signs suggest that respondents gave socially desirable answers in the intervieweradministered modes. A particularity of survey modes is that sample compositions are often heterogeneous. If the selective process is correlated with model elements, such as traits, it can bias invariance tests and decrease fit. We illustrate available options to adjust for this problem, for example propensity score methods or covariate adjustment, all based on the use of auxiliary variables, such as socio-demographics. In conclusion, self-administered modes seem to produce measurements of lower quality than interviewer-administered modes with respect to random error and systematic bias. Modes may thus affect both, the error variance and bias of an estimate. An effect can be suspected particularly between interviewer and non-interviewer modes.
Conference Homepage (abstract)
Web survey bibliography - Schouten, B. (20)
- Adaptive survey designs to minimize survey mode effects – a case study on the Dutch Labor Force...; 2016; Calinescu, M.; Schouten, B.
- Tailored fieldwork design to increase representative household survey response: an experiment in the...; 2015; Luiten, A.; Schouten, B.
- Coding Surveys on their Item Characteristics: Reliability Diagnostics; 2015; Bais, F.; Schouten, B.; Toepoel, V.
- Selection error in single- and mixed mode surveys of the Dutch general population; 2015; Hox, J., Klausch, L. T., Schouten, B.
- Evaluating mixed-mode redesign strategies against benchmark surveys: the case of the Crime Victimization...; 2014; Klausch, L. T., Hox, J., Schouten, B.
- The impact of contact effort on mode-specific selection and measurement bias; 2014; Schouten, B., van der Laan, J., Cobben, F.
- The use of within-subject experiments for estimating measurement effects in mixed-mode surveys ; 2014; Klausch, L. T., Schouten, B., Hox, J.
- The impact of contact effort and interviewer performance on mode-specific nonresponse and measurement...; 2014; Schouten, B., Cobben, F., van der Laan, J., Arends, J.
- Measurement Effects of Survey Mode on the Equivalence of Attitudinal Rating Scale Questions; 2013; Klausch, L. T., Hox, J., Hox, J., Schouten, B.
- Adaptive survey designs to minimize survey mode effects. A case study on the Dutch Labour Force Survey...; 2013; Calinescu, M., Schouten, B.
- Optimizing quality of response through adaptive survey designs; 2013; Schouten, B., Calinescu, M., Luiten, A.
- Mode Effects in Mixed-Mode Surveys: Prevention, Diagnostics, and Adjustment 1; 2013; de Leeuw, E. D., Dillman, D. A., Schouten, B.
- Estimating Measurement Effects of Survey Modes From Between and Within Subject Designs; 2013; Klausch, L. T., Hox, J., Schouten, B.
- Mixed-mode including web: Recent developments at Statistics Netherlands; 2013; Luiten, A., Schouten, B.
- Disentangling Mode-Specific Selection and Measurement Bias in Social Surveys; 2012; Buelens, B., van der Laan, J., Schouten, B., Klausch, L. T., van der Brakel, J., Burger, J.
- Assessing Measurement Equivalence and Bias of Questions in Mixed-mode Surveys Under Controlled Sample...; 2012; Klausch, L. T., Hox, J., Schouten, B.
- The Representativity of Web Surveys of the General Population compared to Traditional Modes and Mixed...; 2012; Klausch, L. T., Schouten, B., Hox, J.
- Disentangling relative mode effects for the web survey mode in the Safety Monitor; 2011; Schouten, B., van de Brakel, J., Buelens, B., Klausch, L. T., van der Laan, J.
- Indicators for the representativeness of survey response; 2009; Schouten, B., Cobben, F., Bethlehem, J.
- Nonresponse Bias in Surveys; 2009; Bethlehem, J., Vehovar, V., Stoop, I., Schouten, B., Shlomo, N., Skinner, C., Montaquila, J.