Notice: the WebSM website has not been updated since the beginning of 2018.

Web Survey Bibliography

Title Apples to Oranges or Gala versus Golden Delicious?: Comparing Data Quality of Nonprobability Internet Samples to Low Response Rate Probability Samples
Source Public Opinion Quarterly (POQ); 81, 1, pp. 213-239
Year 2017
Access date 24.08.2017
Abstract Nonprobability samples have gained mass popularity and
use in many research circles, including market research and some political
research. One justification for the use of nonprobability samples is
that low response rate probability surveys have nothing significant to
offer over and above a “well built” nonprobability sample. Utilizing an
elemental approach, we compare a range of samples, weighting, and
modeling procedures in an analysis that evaluates the estimated bias of
various cross-tabulations of core demographics. Specifically, we compare
a battery of bias related metrics for nonprobability panels, dualframe
telephone samples, and a high-quality in-person sample. Results
indicate that there is roughly a linear trend, with nonprobability samples
attaining the greatest estimated bias, and the in-person sample, the
lowest. Results also indicate that the bias estimates vary widely for the
nonprobability samples compared to either the telephone or in-person
samples, which themselves tend to have consistently smaller amounts of
estimated bias. Specifically, both weighted and unweighted dual-frame
telephone samples were found to have about half the estimated bias
compared to analogous nonprobability samples. Advanced techniques
such as propensity weighting and sample matching did not improve
these measures, and in some cases made matters worse. Implications for
“fit for purpose” in survey research are discussed given these findings.
Year of publication2017
Bibliographic typeJournal article

Web survey bibliography (8390)