Notice: the WebSM website has not been updated since the beginning of 2018.

Web Survey Bibliography

Title A Systematic Review of Studies Investigating the Quality of Data Obtained with Online Panels
Year 2012
Access date 26.06.2012

A core tenet of survey research is that the inferences one makes about the population can only be as good as the quality of the respondents in the sample. However, with declines in probability sample response rates and increases in non-probability Internet-based research, researchers have found it increasingly difficult to agree on the quality of a survey sample. Contributing to this difficulty is the variety of research studies that have evaluated the quality of survey data derived from probability-based and non-probability-based sources and the effectiveness of statistical methods to reduce error in data from
these sources. Specifically, some research has documented a greater average error among non-probability samples relative to probability samples (Chang & Krosnick, 2009; Yeager et al., in press), while other research has found few or small differences between the two. Other research has pointed to greater variability in results from surveys non-probability samples of Internet volunteers. For instance, Dedeker (2006) conducted the same study twice on the same Internet survey panel and reached two different business conclusions. An additional study found five to ten times greater variability in error among a sample of seven Internet surveys of non-probability samples versus seven probability sample surveys (Yeager et al., in press). Similarly-sized variability was found in the National Dutch Online Panel Comparison Study. Relatedly, statistical methods such as post-stratification survey weighting have inconsistent effects on non-probability sample surveys, and in some cases increase survey error. It is critically important to synthesize the survey accuracy studies summarized above as well as others. The present study will evaluate the evidence from more than 45 different studies have assessed the accuracy of non-probability sample surveys and the effectiveness of methods to improve their accuracy,
with the aim of helping researchers and consumers to have more informed expectations about data quality in their surveys.

Access/Direct link

Conference Homepage (abstract)

Year of publication2012
Bibliographic typeConferences, workshops, tutorials, presentations

Web survey bibliography - The American Association for Public Opinion Research (AAPOR) 67th Annual Conference, 2012 (50)