Weighting in Web surveys
In classical sampling literature the weights either get only indirect attention (e.g. Kish, Cochran, Kalton, Hansen&Hurwitz) as inverse of inclusion probabilities, or, they are an implicit part of the modelling approach (e.g. Sardnal&Swenson&Wretman). So far, no textbook exists yet that would concentrate on weighting alone.
On the other hand, in paractice of survey sampling, weighting often becomes a central issue. Practitioners developed various robust procedures, in particular when sampling procedures deviate from probability selection, or, when we face nonresponse problems.
With weighting. typically, after adjusting for inclusion probabilities, the data are also adjusted to population socio-demographic controls. Various weighting procedures exsist, however, the differences in results are usually very small. Typically, the effects of weighting corrections do not remove the bulk of the biases.
Weighting in Web surveys follows the general principles of survey sampling. However, due to extensive use of non-probability samples, the pressure to elaborate their inferential potentials is extremely high. The issue of weighting thus obtained a seemingly new dimension with Web surveys. Despite that, there is actually nothing really new here: the quest to replace expensive probability samples with cheap non-probability ones existed from the very beginign of the survey sampling profession.
The orthodox science of survey sampling clearly sticks to the fact that only probability (scientific) sampling enables statistical inference from sample to the population. For that, we have to know in advance the inclusion probabilities for all units in the population. Otherwise we cannot calculate sampling variance and confidence intervals. From this point of view, there is tus not much to add to what Leslie Kish said on quota samples, commenting that this is basically the issue of an art and not anymore of a science.
Besides of huge commercial pressures to validate Web non-probability samples, there seems to be nothing really new with respect to weighting. The only exception might be the propensity score weighting, initiated in 80’s (Rosenbaum&Rubin 1984) for the analysis of the observational studies and adapted to in 90’s by methodologists from the marketing industry. Particularly, the US company Harris Interactive has often reported successful applications of this approach. However, the profound scientific elaboration and the actual added value of this procedure (compared to other weighting strategies) is still somehow ambiguous.
More:
- ISI 2005 Ivited session: The inferential potencials of non-probability surveys.
- Kish, Leslie (1998): ''Quota sampling: Old Plus New Thought.''
- Rosenbaum, Rubin (1984): ''Reducing bias in observational studies using sub-classification in thje propensity score.''
- Terhanian, G. Bremer, J. (2005): ''Creative Applications of Selection Bias Modelling in Market Research.'' 55th ISI Session 2005.
- Vehovar, V., Lozar, Manfrega, K., Batagelj, Z. (1999): ''Web Surveys: Can The Weighting Solve The Problem?'' Proceedings of the Survey Research Methods Section, American Statistical Association, 1999, pp. 962-967.
- Couper, M.p. (2000): ''Web surveys: A review of issues and approaches.''
This article has been read 245 times
