Web Survey Bibliography
Title Web survey response rates compared to other modes - A meta-analysis
Source General Online Research Conference (GOR), 2005;The American Association for Public Opinion Research (AAPOR) 60th Annual Conference, 2005;
Access date 30.03.2005
Abstract One of the questions when discussing the usefulness of Web surveys is whether they gain the same response rates compared to other survey modes. Anecdotal literature reviews suggest that in general, the Web survey response rates are considerably lower. However, such unsystematically synthesized evidence might be highly misleading. As an alternative, systematic meta-analytic procedures synthesizing controlled experimental mode comparisons could provide accurate answers, but to the best of our knowledge, these studies have not been conducted so far. To overcome this gap, we conducted a meta-analysis of 36 published and unpublished experimental comparisons between Web and other survey modes. Due to the fact that response rates proportion differences tend to overestimate the heterogeneity of effects, we took the natural log of the odds-ratio as our effect size measure. To estimate the study population effect under a random effects model assumption, we firstly computed the mean effect size weighted for the inverse effect size variance as suggested by Lipsey and Wilson (2001). Subsequently, a homogeneity analysis and a meta-regression were performed. To predict the response rate variance on the effect size measure, the following study descriptors were included as independent variables: type of alternative mode (e.g., eMail, CATI, mail, fax, IVR), year of study, type of the target population (e.g., students, general population, etc.), sponsorship (academic, government, commercial), and the implementation procedures (incentives, No of contacts, contact mode). While the weighted mean effect size indicated a slight advantage for other than Web modes, the 95% confidence interval around this mean effect size estimate included zero. Thus, this results suggests that there is no systematic response rate differences between Web surveys and other modes. A homogeneity test revealed that the 36 effect sizes are homogeneous, indicating that the study descriptors are not systematically related to the effect size variability. This result was further supported by a meta-regression with the study descriptors included as independent variables, none of them exerting a significant predictive effect on the effect size variability. Taken together, our meta-analysis highlights that the common assumption of lower response rates for Web surveys compared to other modes does not hold true if scrutinized with the aid of meta-analytic research synthesis procedures, taking into account experimentally controlled primary studies. From a practical point of view, these results might contribute to improve the reputation of Web surveys as one survey mode of comparable data quality with respect to response rates.
Bibliographic typeConferences, workshops, tutorials, presentations
Year of publication2005
Web Survey Bibliography - General Online Research Conference (GOR) 2005 (41)
- The use of internet surveys for the evaluation of mobility information systems; 2005; Baeumer, M.
- Translucent is the mind of the web user; 2005; Britschgi, M.
- Persuasiveness on the Internet: Factors influencing participation in Web survey; 2005; Zajac, J.
- Conjoint-Analysen im Internet. Zu Risiken und Nebenwirkungen; 2005; Melles, T., Daiber, A.
- TV research Online: The strenght of the Eco-Symbols; 2005; Mocigemba, D.
- Options of automating internet-based Career-counselling - an example; 2005; Montel, C.
- Online Interviews as an Instrument of the Development of Organizations. A Meta-Analysis of Online Surveys...; 2005; Poetschke, M., Simonson, J.
- Ein online Experiment: Varianten der online Fragebogengestaltung und ihr Einfluss auf die Bearbeitungseffizienz...; 2005; Sabin, S., Jirasko, M.
- The Quality of Ego-Centered Network Data: A comparison of online versus offline data collection; 2005; Snijders, C., Matzat, U.
- Data Quality in On-line Surveys; 2005; Stangl, A.
- Do not touch data from late-received online questionnaires: Analysis of late responder in a closed-pool...; 2005; Stieger, S., Voracek, M.
- Dynamic Interviewing Program (DIP): Online- Interviews via ICQ; 2005; Stieger, S., Reips, U. -D.
- Effects of online surveys on data quality; 2005; Taddicken, M., Reiser, S.
- Challenges in conducting longitudinal randomized controlled trials online - The story of evaluating...; 2005; Davidovich, U.
- The spam problem from the point of view of the working group NEON; 2005; Escher, C., von Heesen, B., Lipsmeier, G., Dallwitz-Wegner, D., Daiber, A.
- Onlinebefragungen kostenlos - Der "Befrager"; 2005; Hansen, C.
- CIAO - Online Research-Barometer; 2005; Irmer, C.
- Analysing and visualising logfiles: the Individualised SiteMap tool ISM; 2005; Kralisch, A., Berendt, B.
- www.eccor-panel.org - Ein deutsch-polnisches Online-Access-Panel als Messstation kultureller Werte und...; 2005; Krawietz, M.
- Using mobile technology to increase responses of people with an active lifestyle; 2005; Neubarth, W., Kaczmirek, L., Bosnjak, M., Bandilla, W.
- Online graphic rating scales; 2005; Siegmar, O.
- Optimizing Prize Values in Web Surveys: Further Examination of the Immediacy Effect; 2005; Tuten, T. L., Galesic, M., Bosnjak, M.
- Personalization, authentication and self-disclosure in self-administered Internet surveys; 2005; Joinson, A. N., Woodley, A., Reips, U. -D.
- Progress indicators in filter based surveys. Individual and dynamic calculation methods; 2005; Kaczmirek, L., Neubarth, W., Bosnjak, M., Bandilla, W.
- Active User Sampling. Intercept surveys without popups; 2005; Kaczmirek, L., Neubarth, W.
- Case Study: Fraunhofer-Gesellschaft Knowledge Management Online-Survey; 2005; Karapidis, A., Schnalzer, K., Wesoly, M.
- The importance of ensuring anonymity in online research; 2005; Leidenfrost, B., Strassnig, B.
- Impact of Topic Salience, Survey Length, and Incentive Type on Response-Rates in a Web-Based Survey:...; 2005; Lindner, S., Marcus, B., Bosnjak, M., Pilischenko, S., Melzer, S., Schuetz, A.
- Datacollection and Datahandling with the crosssite Sample of the OnSite-Survey in the AGOF Project "...; 2005; Lipsmeier, G.
- Lessons From An Online Experiment; 2005; Loewe, G., Read, D., Airoldi, M.
- Web survey response rates compared to other modes - A meta-analysis; 2005; Lozar Manfreda, K., Bosnjak, M., Haas, I., Vehovar, V.
- Online-Studien im Bereich b2b - Erfahrungen aus der Luftfahrtbranche; 2005; Althoff, S.
- Online Surveys in Travel Behaviour Research; 2005; Baeumer, M.
- Effects of two innovative techniques to apply incentives in online access panels; 2005; Bosnjak, M.
- The quality of internet survey data; 2005; Denscombe, M.
- Measurement of Internet usage - Survey and problems; 2005; Fisch, M.
- Online oder CATI? Ein komplexes Methodenexperiment; 2005; Follmer, R.
- From short message service (SMS) - questionnaires to mobile Internet surveys: Integrating the mobile...; 2005; Friedrich-Freksa, M., Liebelt, M.
- Visual analogue scales in online surveys; 2005; Funke, F.
- Visual Analogue Scales in Internet-based Surveys; 2005; Gnambs, T.
- Meta-Analyses on Contingent versus Unconditional Incentives; 2005; Goeritz, A.