Web Survey Bibliography
In a world where we are concerned about the survey taking behaviour of online panellists, it may seem strange to know that one of their biggest complaints is that they rarely get to do surveys!
On our US panel only 1 in 8 survey starts results in a completed interview. The rest comprise Quota Fulls and Screen Outs. Better project management, and more time in field, can reduce the number of Quota Fulls, but the level of Screen Out is by design. The evidence from our European and APAC panels is that the problem is somewhat less, but is increasing.
The result of this excessive screening out is a shortened panellist lifetime, increased dissatisfaction with the research process, and, at worst, an incentive to cheat on surveys.
An alternative approach to the current invitation paradigm, is to decide which survey a panellist should be presented with as they arrive to take a survey, in response to a generic invitation. By presenting a (short) set of screener questions and adding this to the already known panellist profile, a real time profile of the respondent with respect to the currently open surveys can be built, and the panellists then routed to the most appropriate open survey.
By utilising such an approach we expect to improve data quality, but at what cost? The biases introduced by a survey router (as such systems are generally known) are complex, subtle and entirely dependent on the interplay of the surveys currently in the system. Our research takes the form of a Trial by Ordeal. We have re-created a random 50 projects conducted in the past and placed them in a survey router situation. In addition to describing the process and the potential for the introduction of bias, we will examine data distributions to assess how much effect any biases introduce to the data itself and, through simulation, estimate the size of such biases. We will also present findings on data quality – validity and reliability - comparing the two approaches.
Conference Homepage (abstract)
Web survey bibliography - General Online Research Conference (GOR) 2010 (17)
- Testing the Applicability of Respondent Driven Sampling as an Online Research Method to Sample Hidden...; 2010; Pajak, D.
- Seriousness Checks are Useful to Improve Data Validity in Online Research; 2010; Diedenhofen, D., Aust, F., Ullrich, S., Musch, J.
- Enrichment of Qualitative Research through Online Approaches: New Insights due to Online CoCreation...; 2010; Krischke-Ramaswamy, M., Knorr, H.
- Developing and Evaluating a Student Online Panel.; 2010; Stiglbauer, B., Gamsjäger, M., Gnambs, T., Batinic, B., Altrichter, H.
- Online Access Panels: A detailed look at different Ways of Entering, their Costs and Participation Behavior...; 2010; Führer, R., Keusch, F.
- Eye Tracking and Cognitive Interviewing: Steps to improve online questionnaires; 2010; Tries, S., Sattelberger, S.
- Trial by Ordeal, a medieval approach to a modern day problem; 2010; Cape, P., Cavallaro, K.
- How new engagement techniques and question approaches are revolutionizing online research data gathering...; 2010; Puleston, J.
- Social Networking Sites: New approaches for Online-Panels?; 2010; Drosdow, M., Geißler, H.
- The Impact of Visual and Functional Design Elements in Online Survey Research; 2010; Hammen, K.
- Theoretical model of context-sensitive mobile methods; 2010; Maxl, E.
- Can a professional questionnaire layout make up for a boring topic? The mediating role of topic interest...; 2010; Keusch, F., Mayerhofer, W., Jungreithmaier, S., Weilbuchner, N., Fuehrer, R., Kling, H.
- Using Propensity Score Weighting to Reduce Bias of a Swiss Market Research Web Panel; 2010; Wiegand, G., Jella, H., Beat, H., Stefan, L.
- Potentials and Constraints of Propensity Score Weighting to Improve Web Survey Quality; 2010; Steinmetz, S., Tijdens, K.
- Selection Bias in Web Surveys and the Use of Propensity Scores in Forecasting the Result of the 2009...; 2010; Musch, J., Ullrich, S., Diedenhofen, D.
- Breakoff in Web Surveys of the German Longitudinal Election Study (GLES); 2010; Blumenstiel, J. E., Roßmann, J., Steinbrecher, M.
- The longitudinal effect of incentives on participation and data quality in online panels; 2010; Neumann, B. P., Goeritz, A.