Web Survey Bibliography
Relevance and research question:
Self-selected survey participants typically differ from the general population with regard to demographic variables that are important for voting behaviour. For example, elderly persons and persons with a low educational level are usually underrepresented. To forecast the result of elections based on such non-probability samples, it is therefore necessary to statistically adjust for selection bias. We investigated whether the validity of an election forecast can be improved using propensity scores which rely on demographic characteristics to model the propensity to participate in web surveys.
Methods and data:
In the week prior to the 2009 German federal election to the Bundestag/Lower House of the Parliament, we surveyed a large self-selected sample of potential voters. Recording several demographic variables that are presumably relevant for political preferences, we also asked the participants to indicate their voting intention in the upcoming election. We then computed predictions for the outcome of the election using propensity score adjustments based on official Mikrozensus data. We compared these predictions with the outcome of the election, and also tested whether propensity adjusted Web-based convenience samples allow for a prediction that is competitive with probability sample-based benchmark estimates.
Results:
We found that propensity score adjustments were able to considerably improve the predictive validity of an Internet-based survey of a non-random sample of potential voters, leading to a much better forecast of the election outcome. Our results thus document the usefulness of propensity scores to improve the validity of election forecasts. However, we also noted several limitations of the approach.
Added value:
Propensity score adjustments are increasingly being used to control for selection bias, but there are few opportunities to validate this approach against external criteria. Forecasting the outcome of an election is a rare but important exception, because adjusted and unadjusted predictions can be directly compared to an election’s official result.
Conference Homepage (abstract)
Web survey bibliography - General Online Research Conference (GOR) 2010 (17)
- Testing the Applicability of Respondent Driven Sampling as an Online Research Method to Sample Hidden...; 2010; Pajak, D.
- Seriousness Checks are Useful to Improve Data Validity in Online Research; 2010; Diedenhofen, D., Aust, F., Ullrich, S., Musch, J.
- Enrichment of Qualitative Research through Online Approaches: New Insights due to Online CoCreation...; 2010; Krischke-Ramaswamy, M., Knorr, H.
- Developing and Evaluating a Student Online Panel.; 2010; Stiglbauer, B., Gamsjäger, M., Gnambs, T., Batinic, B., Altrichter, H.
- Online Access Panels: A detailed look at different Ways of Entering, their Costs and Participation Behavior...; 2010; Führer, R., Keusch, F.
- Eye Tracking and Cognitive Interviewing: Steps to improve online questionnaires; 2010; Tries, S., Sattelberger, S.
- Trial by Ordeal, a medieval approach to a modern day problem; 2010; Cape, P., Cavallaro, K.
- How new engagement techniques and question approaches are revolutionizing online research data gathering...; 2010; Puleston, J.
- Social Networking Sites: New approaches for Online-Panels?; 2010; Drosdow, M., Geißler, H.
- The Impact of Visual and Functional Design Elements in Online Survey Research; 2010; Hammen, K.
- Theoretical model of context-sensitive mobile methods; 2010; Maxl, E.
- Can a professional questionnaire layout make up for a boring topic? The mediating role of topic interest...; 2010; Keusch, F., Mayerhofer, W., Jungreithmaier, S., Weilbuchner, N., Fuehrer, R., Kling, H.
- Using Propensity Score Weighting to Reduce Bias of a Swiss Market Research Web Panel; 2010; Wiegand, G., Jella, H., Beat, H., Stefan, L.
- Potentials and Constraints of Propensity Score Weighting to Improve Web Survey Quality; 2010; Steinmetz, S., Tijdens, K.
- Selection Bias in Web Surveys and the Use of Propensity Scores in Forecasting the Result of the 2009...; 2010; Musch, J., Ullrich, S., Diedenhofen, D.
- Breakoff in Web Surveys of the German Longitudinal Election Study (GLES); 2010; Blumenstiel, J. E., Roßmann, J., Steinbrecher, M.
- The longitudinal effect of incentives on participation and data quality in online panels; 2010; Neumann, B. P., Goeritz, A.