Volunteer opt-in panels dominate the Web survey research today, primarily because of low cost and speed of data collection. However, it is not clear how well the results obtained on such panels represent true population values. On the other hand, there exist large, probabilistically selected panels in both Germany and the US that promise statistically sound inference to the general population. However, such probabilistic panels are much more expensive to establish, maintain, and use, than the panels of volunteers. The research on comparability of the two methods is still lacking.

We conducted equivalent Web-based surveys on volunteer opt-in panels and on probabilistically selected panels of German and US population. The surveys included questions about various characteristics of the participants, such as numeracy and graphical literacy. The surveys also included experiments on the effects of different ways to present risk-related information on understanding and perception of risks.

In this paper, we compare the results from different panels in terms of: 1) estimates for prevalence of different characteristics in the population, 2) the effect sizes obtained in the experiments, 3) data quality, and 4) cost. We discuss the results in particular in respect to value-for-money of the different types of panels.