Web Survey Bibliography
The visual capabilities offered by the internet provide a platform by which magazine readers may be queried about their viewing, noting and recognizing of ad copy appearing in specific magazine issues. However, it is well known that “samples” used in these studies may be subject to substantial bias arising from the non-probability nature of the sample selection process. Furthermore, when correctly computed, the response rates on many internet panels are quite low. In those situations when certain key variables are statistically linked (i.e. strongly correlated) with sample selection bias and key substantive outcomes, these variables may be used to adjust or calibrate these estimates. This is sometimes known as post stratification in traditional full-probability sampling and model-based estimation for model based (non-probability) sampling. In examining a large number of internet samples used to collect data on ad-noting and ad recognition it is has been found that these outcome measures are associated and correlated, to varying degrees, with gender, time spent reading, place of reading, percent of pages opened, and frequency of reading. Furthermore, we have found the distribution of these variables among internet respondents is substantially different from those in traditional full-probability surveys. We have developed a series of sample weighting procedures to remove a substantial amount of the “selection bias” linked to these reading qualities. This bias reduction step results in meaningful changes in readership ad-noting and ad identification. This paper will show, using actual data, how our approach to bias reduction weighting was developed, and how it impacts the outcomes of ad-noting and identification. In our decision to apply these weights we have adopted a standard minimization of mean squared error approach and perspective. That is, any weighting which increases variable random error must be offset with bias reduction. Bias reduction occurs when changes in the survey estimates are observed. Within a single magazine issue, the overall changes in ad noting scores are not typically large. However, there are ads in which noting scores do show substantial change. These changes are consistent with expectations linked to the adjustment measures. Furthermore, while an outside validation of the model based estimates has not been undertaken, our examination of overall impact across magazines is highly consistent with those expected on the basis of the variables involved. Thus, while we do not claim that our results are externally validated, we are comfortable in saying that the adjustments are in the expected direction and appear to make sense.
Conference Homepage (abstract) / (full text)