Relevance & Research Question: Propensity score weighting (PSW) is frequently used to correct for attrition biases in panel surveys. While there is a rich methodological literature on the logic of PSW and studies on its practical application, we face a lack of in-depth discussion on the effects of using PSW to correct for attrition biases in attitudinal, behavioral, and socio-demographic variables. Consequently, we address the questions, first, whether there are differences in attrition biases between different types of variables and, second, whether we can identify patterns in the effects of applying PSW across these types of variables.

Methods & Data: Our analysis draws on data from a seven-wave web-based split-panel survey conducted during the campaign to the 2013 German federal election. The panel is supplemented with cross-sectional surveys that are comparable in terms of sampling and questionnaire. We use these cross-sections to assess attrition biases in the corresponding waves of the panel survey. The propensity score weights are calculated using the predicted propensity of respondents to participate in consecutive panel waves. The estimation of the response propensities draws on the data from the first panel wave. We assess the effect of applying these weights on attrition bias in 48 attitudinal, 38 behavioral, and 27 socio-demographic variables.

Results: Our results show that PSW successfully reduced biases in 72 out of the 113 variables. However, looking at the three types of variables, we find the rate of success to be lowest for behavioral variables compared to socio-demographics and attitudinal variables. Furthermore, the magnitude of the reduction in biases is lower for socio-demographic and behavioral variables compared to attitudinal variables.

Added Value: Our findings suggest –while considering the estimate-specific nature of bias–, first, that biases vary across different types of variables and, second, that the effects of PSW are not homogeneous across these types. Accordingly, we recommend not to restrict evaluations of attrition in a panel survey to a limited set of (socio-demographic) variables, because this may result in an underestimation of the magnitude of biases and an overestimation of the ability of PSW to reduce biases in other (types of) variables.