Web Survey Bibliography
Non-sampling errors, and in particular, those arising from non-response and the measurement process itself present a particular challenge to survey methodologists, because it is not always easy to disentangle their joint effects on the data. Given that factors influencing the decision to participate in a survey may also influence the respondents' motivation and ability to respond to the survey questions, variations in the quality of responses may simultaneously be caused by both non-response bias and measurement error. In this study, we examine factors underlying both kinds of error using data from the 2008 ANES Internet Panel. Using interview and paradata from the initial recruitment survey, we investigate the relationship between recruitment effort (e.g. number of contact attempts; use of refusal conversion efforts), willingness to participate in subsequent panel waves, and the ability and motivation to optimize during questionnaire completion. We find that respondents who were hardest to reach or persuade to participate in the recruitment interview responded to fewer monthly panel surveys overall, and were more likely to stop participating in the panel altogether. They also had higher rates of item non-response in the recruitment interview. Respondents who later stopped participating in the panel were also more likely to give answers of reduced quality in the wave 1 monthly survey (e.g. more midpoint answers, less differentiation between scale points for question batteries, and fewer responses to a check-all-that-apply question format). We then investigated two potential common causes of the observed relation between the propensity to stop participating in the panel and response quality (interest in computers, and ‘need to evaluate’), but neither one fully accounted for it. Interest in computers predicted later panel cooperativeness, while need to evaluate was related both to response quality and the propensity to attrit. Finally, we look at whether the panelists most likely to stop participating in the panel are also more likely to learn shortcutting strategies over time, to reduce the effort needed to complete monthly surveys. We find some support for this hypothesis. We discuss our findings and their implications for the design of future online panels.
Web survey bibliography - In M. Callegaro, R. Baker, J. Bethlehem, A. S. Göritz, J. A. Krosnick and P. J. Lavrakas (eds.): Online Panel Research: A Data Quality Perspective. John Wiley & Sons, Ltd, Chichester, UK (15)
- Validating respondents' identity in online samples; 2014; Baker, R., Miller, C., Kachhi-Jiwani, D., Lange, K., Wilding-Brown, L., Tucker, J.
- The relationship between nonresponse strategies and measurement error; 2014; Malhotra, N., Miller, J. M., Wedeking, J.
- Nonresponse and measurement error in an online panel; 2014; Roberts, C., Allum, N., Sturgis, P.
- Estimating the effects of nonresponses in online panels through imputation; 2014; Zhang, W.
- An empirical test of the impact of smartphones on panel-based online data collection; 2014; Drewes, F.
- Professional respondents in nonprobability online panels; 2014; Hillygus, D. S., Jackson, N. M., Young, M.
- Informing panel members about study results; 2014; Scherpenzeel, A., Toepoel, V.
- Determinants of the starting rate and the completion rate in online panel studies; 2014; Goeritz, A.
- The untold story of multi-mode (online and mail) consumer panels; 2014; McCutcheon, A. L., Rao, K., Kaminska, O.
- Online panels and validity; 2014; Groenlund, K., Strandberg, K.
- Assessing representativeness of a probability-based online panel in Germany; 2014; Struminskaya, B., Kaczmirek, L., Schaurer, I., Bandilla, W.
- A critical review of studies investigating the quality of data obtained with online panels based on...; 2014; Callegaro, M., Villar, A., Yeager, D. S., Krosnick, J. A.
- Online panel research: History, concepts, applications and a look at the future; 2014; Callegaro, M., Baker, R., Bethlehem, J., Goeritz, A., Krosnick, J. A., Lavrakas, P. J.
- Motives for joining nonprobability online panels and their association with survey participation behavior...; 2014; Keusch, F., Batinic, B., Mayerhofer, W.
- Improving web survey quality; 2014; Steinmetz, S., Bianchi, S. M., Tijdens, K. G., Biffignandi, S.