Notice: the WebSM website has not been updated since the beginning of 2018.

Web Survey Bibliography

Title The River Flows: Comparison of Experimental Effect Replicability with Different Sample Sources
Year 2012
Access date 30.06.2012

There are many different sample sources used today in market and opinion research. Based on number of studies being fielded and dollar volume, two of the most common sample sources in web-based research are river samples and non-probability panels. A non-probability panel recruits respondents in a number of different ways to collect together a group of people who will complete a number of surveys over time. While studies employing non-probability panels have an extensive history of use going back to the 1950s, river samples became more prominent in the past 10 years for web-based studies. A river sample recruits respondents on an ad hoc basis using online pop-up ads and other online recruitment techniques. Once river respondents respond to an invitation they are typically directed to an initial survey that screens them with demographic questions and other topic-relevant questions and then passes them over to ongoing surveys based on quota needs of a number of simultaneous live surveys. This study focused on whether experimental effects are similar in both direction and size in both types of samples. Respondents were randomly assigned to either a yes-no grid, a multiple response format (‘select all’), or a household combination grid to determine the purchase of a series of 16 products. We found similar effects for both sample sources in experimental effects – the combination grid had the highest incidence; the multiple response format was associated with lowest incidence. Though relative order of the results were similar, the river sample showed generally higher incidence for all product types across experimental conditions, however, the differences became smaller after controlling for demographic differences and self-rated consumer style (a series of measures to determine consumer orientation). We examine and propose additional analyses that could lead to reductions in differences in sample source outcomes.

Access/Direct link

Conference Homepage (abstract)

Year of publication2012
Bibliographic typeConferences, workshops, tutorials, presentations

Web survey bibliography - The American Association for Public Opinion Research (AAPOR) 67th Annual Conference, 2012 (50)