Notice: the WebSM website has not been updated since the beginning of 2018.

Web Survey Bibliography

Title Improving the Efficiency of Web Survey Experiments
Source The American Association for (AAPOR) 63rd Annual Conference, 2008 & WAPOR 61th Annual Conference, 2008
Year 2008
Access date 02.06.2009

This paper investigates potential improvements that can be made to online survey experiments using pairing and blocking techniques, as opposed to simple random assignment. While random assignment studies hold the assumption that control and treatment groups have the same characteristics, statistical variability causes this to not always be the case in practice. With some frequency, control and treatment groups end up being significantly different on one or more variables of interest, simply by chance. As a solution, matched pair and block experiments create control and treatment groups that are more similar than they would be by purely random allocation. In matched pair experiments, respondents with similar characteristics are “paired” and assigned to separate treatments, thereby ensuring that the control and experimental groups are statistically identical. This design leads to increased efficiency in treatment effect estimates (Randomized blocking is a more general case of this type of design).In a recent YouGov-Polimetrix survey, we implemented survey software that assigns respondent pairs and blocks to separate treatment groups automatically, while avoiding the predictable assignment problem. Using data from this survey, we compare the efficiency between randomized pair, randomized block, and simple random assignment experiments. Additionally, we examine if any types of unexpected bias are introduced by using randomized pair and block designs instead of traditional methods.

Access/Direct link


Year of publication2008
Bibliographic typeConferences, workshops, tutorials, presentations
Full text availabilityAvailable on request