Relevance & Research Question: The use of propensity score weighting to reduce selection bias in nonprobability online panels has increased considerably in recent years. Previous research shows that marginal distributions can quite well be adjusted to a representative reference study using both socio-demographic variables and variables that are closely related to the survey topic. But studies that used propensity score weighting to provide estimates of bivariate and multivariate relationships produced mixed results. Another drawback using survey specific covariates is that the corresponding variables have to be selected anew for each research area. We argue for the use of more basic personality traits as covariates, which are likely to explain the self-selection in nonprobability online panels and that allow the general construction of propensity scores across research areas. In the present study we explore the potential role of ‘Big Five’ personality traits to improve propensity score adjustments.
Methods & Data: We conducted two parallel surveys on voting behavior with identical questions administered via computer-assisted personal interviewing (CAPI) to a probability sample and via internet to a nonprobability sample drawn from a commercial online panel. In a three step adjustment design, we first adjusted the online survey for demographics. In a second and third step we sequentially added personality traits and political attitudes as covariates. After each step we compared univariate, bivariate and multivariate results between the web survey and the representative reference study to test the adjustment performance.
Results: Preliminary analyses suggest that considering the ‘Big Five’ personality dimensions as covariates in calculating propensity scores leads to a significant improvement in adjusting data from nonprobability online panels. Marginal distributions of vote intentions as well as relationships between vote intentions and various determinants of voter decision-making are more similar between the two samples using personality traits as covariates.
Added Value: The present study makes a contribution to the greater problem of biased data through selectivity in nonprobability online panels. Through improvement of the adjustment procedures, disadvantages of nonprobability online panels can be reduced.