Despite concerns about the non-probability basis for web surveys, an increasing number of studies have found that sample estimates from web surveys compare favorably to those from probability surveys. Aside from the inability to apply probability statistics to these web surveys, potential users worry about how reliable are these estimates.
This paper examines one element that may contribute to the representativeness of web surveys, at least in certain circumstances. Who are the participants and non-participants of these now ubiquitous sources used in population estimates and how similar are they to the population they are meant to represent?
The purpose of this paper is to investigate the actual coverage of Internet panels, generally, among Americanadults. Rather than look at the composition of any one online survey, we consider the characteristics of the population who participate in any web based panel surveys. Since web panels frequently “partner” with other web panel organizations in order to generate samples that are larger, more diverse or more specific than what is available within their panel members, the population who participate in any web panels is more critical to evaluating coverage error and bias of this form of survey than specific coverage and bias errors in individual panels.
A national probability survey of adults was conducted using a dual frame, random digit dialing sample determined participation in Internet survey panels and frequency of participation. Using several classifications for web panelist, we compare the population characteristics of the universe of web panelists compared to the general public. These findings allow us to characterize the degree of coverage and bias associated with the general population of web panelists. We believe that this is the first step in understanding the issues associated with the representativeness and non-representativeness of web survey findings.