Notice: the WebSM website has not been updated since the beginning of 2018.

Web Survey Bibliography

Title Evaluating the Potential Contributions of a Web-based Convenience Sample to the Accuracy of a Probability Sample
Source The American Association for (AAPOR) 63rd Annual Conference, 2008 & WAPOR 61th Annual Conference, 2008
Year 2008
Access date 01.06.2009

We describe a methodology for combining a convenience sample with a probability sample in a way that seeks to minimize the total mean squared error (MSE) of the resulting estimator. We then explore the properties of the resulting composite estimator, a linear combination of the convenience and probability sample estimators with weights that are a function of bias. We discuss the estimator’s properties in the context of web-based convenience sampling. Our analysis demonstrates that the use of a convenience sample to supplement a probability sample for cost-effective improvements in the MSE of estimation may be practical only under very limited circumstances. First, the bias remaining, after steps are taken to reduce it, must be quite small, equivalent to no more than 0.02-0.1 standard deviations (about one to five percentage points for a dichotomous outcome). Second, the probability sample should contain at least 1000-10,000 observations in order to effectively estimate bias. Third, it must be inexpensive and feasible to collect at least thousands (and probably tens of thousands) of web-based convenience observations. The convenience sample may be a useful supplement in a large survey where the primary goal is estimates within smaller domains if one is willing to assume that global bias estimates from the full sample also apply to smaller domains. The conclusions about the limited usefulness of convenience samples with estimator bias of more than 0.1 standard deviations can be shown to apply more generally.

Access/Direct link

Homepage (abstract)

Year of publication2008
Bibliographic typeConferences, workshops, tutorials, presentations
Full text availabilityAvailable on request

Web survey bibliography (183)