Web Survey Bibliography
Title Web survey response rates compared to other modes - A meta-analysis
Source General Online Research Conference (GOR), 2005;The American Association for Public Opinion Research (AAPOR) 60th Annual Conference, 2005;
Access date 30.03.2005
Abstract One of the questions when discussing the usefulness of Web surveys is whether they gain the same response rates compared to other survey modes. Anecdotal literature reviews suggest that in general, the Web survey response rates are considerably lower. However, such unsystematically synthesized evidence might be highly misleading. As an alternative, systematic meta-analytic procedures synthesizing controlled experimental mode comparisons could provide accurate answers, but to the best of our knowledge, these studies have not been conducted so far. To overcome this gap, we conducted a meta-analysis of 36 published and unpublished experimental comparisons between Web and other survey modes. Due to the fact that response rates proportion differences tend to overestimate the heterogeneity of effects, we took the natural log of the odds-ratio as our effect size measure. To estimate the study population effect under a random effects model assumption, we firstly computed the mean effect size weighted for the inverse effect size variance as suggested by Lipsey and Wilson (2001). Subsequently, a homogeneity analysis and a meta-regression were performed. To predict the response rate variance on the effect size measure, the following study descriptors were included as independent variables: type of alternative mode (e.g., eMail, CATI, mail, fax, IVR), year of study, type of the target population (e.g., students, general population, etc.), sponsorship (academic, government, commercial), and the implementation procedures (incentives, No of contacts, contact mode). While the weighted mean effect size indicated a slight advantage for other than Web modes, the 95% confidence interval around this mean effect size estimate included zero. Thus, this results suggests that there is no systematic response rate differences between Web surveys and other modes. A homogeneity test revealed that the 36 effect sizes are homogeneous, indicating that the study descriptors are not systematically related to the effect size variability. This result was further supported by a meta-regression with the study descriptors included as independent variables, none of them exerting a significant predictive effect on the effect size variability. Taken together, our meta-analysis highlights that the common assumption of lower response rates for Web surveys compared to other modes does not hold true if scrutinized with the aid of meta-analytic research synthesis procedures, taking into account experimentally controlled primary studies. From a practical point of view, these results might contribute to improve the reputation of Web surveys as one survey mode of comparable data quality with respect to response rates.
Year of publication2005
Bibliographic typeConferences, workshops, tutorials, presentations
Web Survey Bibliography - Bosnjak, M. (61)
- Online Survey Participation via Mobile Devices: Findings From Seven Access Panel Studies; 2013; Bosnjak, M., Poggio, T., Funke, F.
- Sample composition discrepancies in different stages of a probability-based online panel; 2013; Bosnjak, M., Haas, I., Galesic, M., Kaczmirek, L., Bandilla, W., Couper, M. P.
- Mobile Survey Participation Rates in Commercial Market Research: A Meta-Analysis; 2012; Bosnjak, M., Poggio, T., Becker, K. R., Funke, F., Wachenfeld, A., Fischer, B.
- Self-administered mobile surveys; 2011; Bosnjak, M.
- Use of new technologies in social research: Self-administered mobile surveys; 2011; Bosnjak, M.
- Self-administered mobile surveys: Usability and (non)participation; 2010; Scherrer, S., Bosnjak, M.
- Internet surveys; 2010; Couper, M. P., Bosnjak, M.
- Understanding the Willingness to Participate in Mobile Surveys: Exploring the Role of Utilitarian, Affective...; 2010; Bosnjak, M., Metzger, G., Graef, L.
- Potenzial selbstadministrierter mobiler Befragungen: Erfahrungen aus einer empirischen Studienreihe; 2009; Bosnjak, M.
- Web Surveys; 2009; Bosnjak, M.
- Potential of self-administered mobile surveys; 2009; Bosnjak, M.
- Strategies for Increasing Response Rates in Web Surveys and their Differential Effects on Sample Composition...; 2009; Haas, I., Bosnjak, M., Bandilla, W., P., Galesic, M.Couper, M. P.
- Understanding the willingness to participate in mobile surveys: Exploring the role of hedonic, affective...; 2009; Bosnjak, M., Metzger, G.
- Measurement options, measurement error, and usability in mobile surveys; 2009; Pferdekämper, T., Bosnjak, M., Metzger, G.
- Effects of Questionnaire Length on Participation and Indicators of Response Quality in a Web Survey; 2009; Galesic, M., Bosnjak, M.
- Prenotification in Web-Based Access Panel Surveys: The Influence of Mobile Text Messaging Versus E-Mail...; 2008; Bosnjak, M., Neubarth, W., Couper, M. P., Bandilla, W., Kaczmirek, L.
- Web Surveys versus Other Survey Modes – A Meta-Analysis Comparing Response Rates; 2008; Lozar Manfreda, K., Bosnjak, M., Berzelak, J., Haas, I., Vehovar, V.Berzelak, N.
- Compensating for low topic interest and long surveys: A field experiment on nonresponse in Web surveys...; 2007; Marcus, B., Bosnjak, M., Lindner, S., Pilischenko, S., Schuetz, A.
- Nonresponse bei Web-Befragungen im Vergleich zu anderen Befragungsmodi - Eine Meta-Analytische Synthese...; 2006; Lozar Manfreda, K., Bosnjak, M., Berzelak, J., Haas, I., Vehovar, V.Berzelak, N.
- Beyond response rates: Effects of different (Web-) survey implementation procedures on sample composition...; 2006; Bosnjak, M., Marcus, B.; Schuetz, A.; Lindner, S.; Pilischenko, S.
- Response Biases in Online Surveys; 2006; Galesic, M., Bosnjak, M.
- Personality traits and participation in an online access panel; 2006; Galesic, M., Bosnjak, M.
- A meta-analysis of response rates in Web surveys compared to other survey modes; 2005; Lozar Manfreda, K., Bosnjak, M., Haas, I., Vehovar, V.
- Prenotification in online access panel surveys: E-mail versus mobile text messaging (SMS); 2005; Neubarth, W., Bosnjak, M., Bandilla, W., Couper, M. P., Kaczmirek, L.
- Did You Get The Message? Using E-Mail and SMS for Prenotification in Web Surveys; 2005; Neubarth, W., Bosnjak, M., Bandilla, W., Couper, M. P., Kaczmirek, L.
- Value and Timing Strategies in Prize Draws: A Further Examination of the Immediacy Effect in Web Surveys...; 2005; Tuten, T. L., Galesic, M., Bosnjak, M.
- Unit (non)response in Web-based access panel surveys: An extended planned-behavior approach; 2005; Bosnjak, M., Tuten, T. L., Wittmann, W. W.
- Using mobile technology to increase responses of people with an active lifestyle; 2005; Neubarth, W., Kaczmirek, L., Bosnjak, M., Bandilla, W.
- Optimizing Prize Values in Web Surveys: Further Examination of the Immediacy Effect; 2005; Tuten, T. L., Galesic, M., Bosnjak, M.
- Progress indicators in filter based surveys. Individual and dynamic calculation methods; 2005; Kaczmirek, L., Neubarth, W., Bosnjak, M., Bandilla, W.
- Impact of Topic Salience, Survey Length, and Incentive Type on Response-Rates in a Web-Based Survey:...; 2005; Lindner, S., Marcus, B., Bosnjak, M., Pilischenko, S., Melzer, S., Schuetz, A.
- Web survey response rates compared to other modes - A meta-analysis; 2005; Lozar Manfreda, K., Bosnjak, M., Haas, I., Vehovar, V.
- Effects of two innovative techniques to apply incentives in online access panels; 2005; Bosnjak, M.
- Progress Indicators in Filter Based Surveys: Computing Methods and their Impact on Drop Out; 2004; Kaczmirek, L., Neubarth, W., Bosnjak, M., Bandilla, W.
- Feasibility of a random sample; 2004; Neubarth, W., Kaczmirek, L., Bandilla, W., Bosnjak, M.
- Effects of Immediate versus Delayed Notification of Prize Draw Results and Announced Survey Duration...; 2004; Tuten, T. L., Galesic, M., Bosnjak, M.
- Mode-Effects in Pre-Recruited Panels of Full Population?; 2004; Bandilla, W., Bosnjak, M., Altdorfer, P., Lohmann, H.
- Web-based surveys in market and social research – Usage and needs of different user groups in...; 2004; Kaczmirek, L., Bosnjak, M., Bandilla, W., Auer, T.
- Stereotypic Response Patterns within Matrix Questions in Web Surveys; 2004; Gockenbach, S., Bosnjak, M., Goeritz, A.
- Web-based surveys in market and social research - usage and needs of different user groups in the EU; 2004; Kaczmirek, L., Bosnjak, M., Bandilla, W., Auer, T.
- Prepaid and promised incentives in Web surveys - An experiment; 2003; Bosnjak, M., Tuten, T. L.
- Survey Administration Effects? A Comparison of Web-Based and Traditional Written Self-Administered Surveys...; 2003; Bandilla, W., Bosnjak, M., Altdorfer, P.
- Online Social Sciences; 2002; Batinic, B., Reips, U. -D., Bosnjak, M.
- Understanding the Willingness to Participate in Online-Surveys - The case of E-mail questionnaires; 2002; Bosnjak, M., Batinic, B.
- Mode-Effects in Web Surveys?; 2002; Bandilla, W., Bosnjak, M.
- (Non)Response bei Web-Befragungen; 2002; Bosnjak, M.
- Prepaid and Promised Incentives in Web Surveys: An Experiment; 2002; Bosnjak, M., Tuten, T. L.
- Internet Surveys and Data Quality - A Review; 2002; Tuten, T. L., Urban, D.J., Bosnjak, M.
- Dimensions of Internet Science; 2001; Reips, U. -D., Bosnjak, M.
- Participation in Non-Restricted Web Surveys: A Typology and Explanatory Model for Item-Nonresponse; 2001; Bosnjak, M.