Relevance & Research Question: Attrition is an important methodological challenge to panel surveys (Lynn 2009). Still, there is a remarkable shortage of variables which are associated with both, the propensity of respondents to stay in the panel and the variables of interest. As a result, propensity score weights which are designed to correct for this type of nonresponse frequently yield mixed results.
This paper addresses the question whether paradata can successfully be applied to improve the prediction of attrition in panel Web surveys. Their main advantage is that they are collected as a byproduct of the survey process. However, it is still an open question which paradata can be used to model attrition and to what extent these paradata are correlated with variables of interest (Kreuter and Olson 2013).
Methods & Data: We use logistic regressions to model attrition in a 7-wave panel Web survey and to compute propensity score weights. The models are fitted with sets of socio-demographic, substantial, survey evaluation, and paradata variables. The latter include measures of response times, user agent strings to determine the device used by the respondent, as well as indicators of the respondents’ response behavior. Finally, we use supplemental cross-sectional Web surveys to assess the effectiveness of propensity score weights based on different sets of variables.
Results: Our results show that including paradata significantly improves the prediction of panel attrition. However, the paradata variables do not supersede socio-demographic, survey evaluation and substantial variables, but they complement them. Yet, the paradata are only moderately correlated with variables of interest at best. As a result, including paradata does not significantly improve the effectiveness of propensity score weights.
Added Value: This paper enhances the existing knowledge in several ways: It presents a set of paradata variables and provides empirical tests of their capability to explain attrition. We show that these paradata can successfully be used to create auxiliary data in a cost-efficient way. At the same time, we demonstrate that they do not ultimately help to correct for panel attrition. Thus, we conclude that further research on paradata, panel attrition and its correction is needed.