Web Survey Bibliography
The full set of questions may be essential to fulfilling the survey objectives, yet long survey instruments can be taxing to respondents. As the respondent continues to answer survey questions for an extended period of time, they may not provide the same level of consideration to all survey questions – providing answers with greater measurement error towards the end of the survey. There is little empirical evidence to support the tradeoff between survey length and measurement error, leaving survey practitioners to rely on expectations to support improvement of survey estimates using a shorter survey. Furthermore, empirical research is in dire need of methods to address the challenge of collecting the full set of variables while minimizing the survey length. One potential solution is the use of split or matrix questionnaire design in which the survey is modularized and respondents are randomly assigned to receive a different subset of the survey modules. In this way, no questions are asked so late in the survey that the measurement error properties of the data are compromised. The full sets of variables are then multiply-imputed for all respondents. This method provides a tradeoff between measurement error in the long survey and increased variances of estimates in the matrix design. We pose two research questions:
1. Whether there is greater measurement error when questions are asked late in the survey, and
2. Whether a matrix questionnaire design can provide estimates with lower total error (MSE).
We use data from a web survey experiment in which a set of questions were asked either early or late in the instrument. We found strong evidence for greater measurement error when the questions were asked late in the survey. This paper will present an evaluation of the use of a matrix questionnaire design instead of the full survey.
Conference Homepage (abstract)
Web survey bibliography - Peytchev, A. (13)
- Multiple Sources of Nonobservation Error in Telephone Surveys: Coverage and Nonresponse; 2011; Peytchev, A.; Carley-Baxter, L. R.; Black, M. C.
- Developing and Implementing Adaptive Total Design (ATD); 2011; Carley-Baxter, L. R., Mitchell, S., Peytchev, A., Day, O.
- Matrix Questionnaire Design to Reduce Measurement Error; 2011; Peytchev, A., Peytcheva, E.
- Increasing Respondents' Use of Definitions in Web Surveys; 2010; Peytchev, A., Conrad, F. G., Couper, M. P., Tourangeau, R.
- Experiments in Mobile Web Survey Design; 2008; Peytchev, A., Hill, C.
- Coverage Bias in Surveys Excluding Cell Phone Only Adults: Evaluation of Bias and Effectiveness of Post...; 2008; Peytchev, A., Carley-Baxter, L. R., Black, M. L.
- Experiments in Visual Survey Design for Mobile Devices; 2008; Peytchev, A., Hill, C.
- Mobile Web Survey Design; 2008; Peytchev, A. Hill, C.
- Following Up Nonrespondents to an Online Weight Management Intervention: Randomized Trial Comparing...; 2007; Couper, M. P., Peytchev, A., Strecher, V., Rothert, K., Anderson, K. J.
- Web Survey Design: Paging versus Scrolling; 2006; Peytchev, A., Couper, M. P., McCabe, S. E., Crawford, S. D.
- Web Survey Design: Paging vs. Scrolling; 2004; Peytchev, A., Crawford, S. D., McCabe, S. E., Conrad, F. G., Couper, M. P.
- Validations in Web-based Surveys; 2003; Crawford, S. D., Peytchev, A.
- Statistical Data Validation in Web Instruments:An Empirical Study; 2002; Peytchev, A., Petrova, E. A.