Web Survey Bibliography

Title When Respondents Know Too Much: Limitations of Web Surveys for Electoral Research
Author Callum, N., J., Sturgis, P.
Source The American Association for Public Opinion Research (AAPOR) 58th Annual Conference, 2003
Year 2003
Access date 06.05.2004
Abstract Internet surveys are attractive because of their low cost and the relative ease of collecting data from large numbers of respondents in a short time. Whilst these benefits inevitably come at a cost (Couper, 2000), the precise nature and extent of these costs are, as yet, not well known. Recently, several studies have examined the utility of online polling for electoral research using essentially an ‘outcome-based’ approach. That is, ‘theoretical’ models are fitted to data from a questionnaire that has been administered both online and in a more traditional mode and differences in model parameters compared.

Such approaches have resulted in generally optimistic conclusions concerning the potential of web methodologies in this area (Gibson and McAllister 2002; Sanders et al 2002). Yet, in many ways, such studies raise more questions than they resolve: given the known problems with coverage, nonresponse and measurement error in Internet polls, the analyst may well ponder on the generality of such findings to other theoretical models involving different variables.

In this paper we too adopt this outcome-based approach to an assessment of the validity of online methods. However, our models deliberately include variables that are likely to be correlated with discrepancies in coverage, nonresponse and measurement between Internet and face-to-face modes. Using data from a face-to-face probability sample and an Internet poll, we compare models containing the same multi-item scales measuring knowledge of politics and science respectively. We examine the impact of survey mode on the ‘difficulty’ of questions, their ability to discriminate between more and less ‘sophisticated’ respondents and the degree of measurement error. We then examine differences in structural parameters between these and other theoretically related variables. We conclude that researchers contemplating online data collection should consider carefully the likely relationship between key theoretical concepts and errors currently inherent in data collected online.
Year of publication2003
Bibliographic typeConferences, workshops, tutorials, presentations
Print

Web Survey Bibliography - The American Association for Public Opinion Research (AAPOR) 58th Annual Conference, 2003 (45)