Web Survey Bibliography
Many research studies collect data through survey questionnaires. In order to enhance the validity of the findings from these studies, it is important for the studies to employ questions thatminimize measurement error. A diverse range of question evaluation methods are available for detecting measurement error in survey questions. Ex-ante question evaluation methods are relatively inexpensive, because they do not require any data collection from actual survey respondents. Other methods require data collection from respondents either in the laboratory or in the field setting. A major gap in the literature is the general lack of evidence that the problems identified by these methods are actually problems as assessed by traditional quality standards such as reliability or validity. Although one would expect these methods to identify questions that produce low quality data, behavior coding is the only technique in the literature that has been consistently shown to predict the reliability and validity survey questions (Dykema, Lepkowski, and Blixt 1997; Hess, Singer, and Bushery 1999). This paper addresses the important gap in the literature about whether the problems identified by question evaluation methods lead to lower quality data. The research in this paper investigates how effectively these methods predict the reliability of survey questions as measured by test-retest correlations obtained from repeated measurements of sample respondents. The study uses question evaluation results from a few ex-ante methods such as expert review and QUAID, laboratory methods such as cognitive interviewing, and field methods such as behavior coding and response latency to predict the reliability of survey questions. In addition, the study evaluates how the results from question evaluation methods relate to other data quality indicators such as item missing data.
AAPOR Homepage (abstract)
Web survey bibliography - Maitland, A. (8)
- Evaluating Grid Questions for 4th Graders; 2016; Maitland, A.
- Web Probing for Question Evaluation: The Effects of Probe Placement ; 2016; Fowler, S.; Willis, G. B.; Moser, R. P.; Townsend, R. L. M.; Maitland, A.; Sun, H.; Berrigan, D.
- Conducting Survey Experiments Using an Online Labor Market ; 2016; Fowler, S.; Willis, G. B.; Moser, R. P.; Townsend, R. L. M.; Maitland, A.; Sun, H.; Ferrer, R.; Berrigan...
- Using eye-tracking to understand how fourth grade students answer matrix items; 2015; Maitland, A.; Sun, H.; Caporaso, A.; Tourangeau, R.; Bertling, J.; Almonte, D.
- Mobile Devices for the Collection of Sensitive Information; 2015; Maitland, A.; Mercer, A. W.; Tourangeau, K.; Williams, Do.
- An Examination of the Relationship Between Pretest Method Results and Data Quality; 2013; Maitland, A.
- How Many Scale Points Should I Include for Attitudinal Questions?; 2009; Maitland, A.
- Should I label all scale points or just the end points for attitudinal questions?; 2009; Maitland, A.