# Web Survey Bibliography

One of the questions when discussing the usefulness of Web surveys is whether they gain the same response rates compared to other survey modes. Anecdotal literature reviews suggest that in general, the Web survey response rates are considerably lower. However, such unsystematically synthesized evidence might be highly misleading. As an alternative, systematic meta-analytic procedures synthesizing controlled experimental mode comparisons could provide accurate answers, but to the best of our knowledge, these studies have not been conducted so far.

To overcome this gap, we conducted a meta-analysis of 36 published experimental comparisons between Web and other survey modes. Due to the fact that response rates proportion differences tend to overestimate the heterogeneity of effects, we took the natural log of the odds-ratio as our effect size measure. To estimate the study population effect under a random effects model assumption, we firstly computed the mean effect size weighted for the inverse effect size variance as suggested by Lipsey and Wilson (2001). Subsequently, a homogeneity analysis and a meta-regression were performed. To predict the response rate variance on the effect size measure, the following study descriptors were included as independent variables: type of alternative mode (e.g., eMail, CATI, mail, fax, IVR), year of study, type of the target population (e.g., students, general population, ...), sponsorship (academic, government, commercial), and the implementation procedures (incentives, No of contacts, contact mode).

While the weighted mean effect size indicated a slight advantage for other than Web modes, the 95% confidence interval around this mean effect size estimate included zero. Thus, this results suggests that there is no systematic response rate differences between Web surveys and other modes. A homogeneity test revealed that the 36 effect sizes are homogeneous, indicating that the study descriptors are not systematically related to the effect size variability. This result was further supported by a meta-regression with the study descriptors included as independent variables, none of them exerting a significant predictive effect on the effect size variability.

Taken together, our meta-analysis highlights that the common assumption of lower response rates for Web surveys compared to other modes does not hold true if scrutinized with the aid of meta-analytic research synthesis procedures, taking into account experimentally controlled primary studies. From a practical point of view, these results might contribute to improve the reputation of Web surveys as one survey mode of comparable data quality with respect to response rates.

# Web survey bibliography - Vehovar, V. (139)

- Implications of disposition codes for monitoring breakoffs in web surveys; 2017; Cehovin, G.; Vehovar, V.
- Overview: Online Surveys; 2017; Vehovar, V.; Lozar Manfreda, K.
- Non-probability Sampling; 2016; Vehovar, V., Toepoel, V., Steinmetz, S.
- Introduction Breakoffs, Questionnaire Breakoffs and Web Questionnaire Length: A Metastudy ; 2016; Cehovin, G.; Vehovar, V.
- Investigating respondent multitasking in web surveys using paradata; 2016; Sendelbah, A.; Vehovar, V.; Slavec, A.; Petrovcic, A.
- The role of word frequencies in detecting unfamiliar terms and their effect on response quality; 2015; Slavec, A.; Vehovar, V.
- Identifying and correcting question-wording problems: the case of Wageindicator; 2015; Slavec, A., Vehovar, V., Tijdens, K. G.
- Web Survey Methodology; 2015; Callegaro, M., Lozar Manfreda, K., Vehovar, V.
- e-Social Science Perspective on Survey Process: Towards an Integrated Web Questionnaire Development...; 2015; Vehovar, V., Petrovcic, A., Slavec, A.
- Questionnaire length and breakoffs in web surveys: a meta study; 2014; Vehovar, V., Cehovin, G.
- Using computational linguistic resources to evaluate and improve survey questions; 2014; Slavec, A., Vehovar, V.
- Investigating respondent multitasking in web surveys: comparison of self-reports and paradata indicators...; 2014; Sendelbah, A., Vehovar, V., Slavec, A.
- WebSM Study: Overview of Features and Pricing of Software Packages in 2014; 2014; Vehovar, V., Cehovin, G.
- WebSM Study: Survey Software in 2014; 2014; Vehovar, V., Cehovin, G., Mocnik, A.
- WebSM draft report: Questionnaire length and breakoffs in web surveys; 2014; Vehovar, V.; Cehovin, G.
- WebSM Study: Survey Software in 2013; 2013; Cehovin, G., Vehovar, V., Poglajen M.
- Web survey software and mobile device support; 2013; Cehovin, G., Vehovar, V.
- WebSM Study: Overview of Features of Software Packages; 2012; Cehovin, G., Vehovar, V.
- WebSM Study: Speed and efficiency of online survey tools; 2012; Cehovin, G., Vehovar, V.
- Data quality of questions sensitive to social-desirability bias in web surveys; 2012; Lozar Manfreda, K., Zajc, N., Berzelak, N., Vehovar, V.
- WebSM Study: Survey software features overview ; 2012; Vehovar, V., Cehovin, G., Kavcic, L., Lenar, J.
- Costs and Errors in Fixed and Mobile Phone Surveys; 2012; Vehovar, V., Slavec, A., Berzelak, N.
- Web Survey Software; 2012; Berzelak, N., Vehovar, V., Slavec, A.
- Surveying general population: What types of experiments are further needed?; 2012; Vehovar, V., Berzelak, N.
- Web panels in Slovenia; 2012; Lenar, J., Vehovar, V.
- Advanced Paradata in Web Surveys: What Can They Tell about the Response Process?; 2012; Vehovar, V., Slavec, A., Berzelak, N., Lozar Manfreda, K.
- Information-Communication Technology Support for Online Surveys: A Need for Integration; 2012; Vehovar, V., Slavec, A., Berzelak, N., Lozar Manfreda, K.
- Global market of web survey software: Status, trends and issues; 2012; Vehovar, V., Slavec, A., Berzelak, N., Lozar Manfreda, K.
- Paradata insight into survey response behaviour: An analysis of a set of hosted web surveys; 2012; Lozar Manfreda, K., Berzelak, N., Vehovar, V.
- Web survey software; 2011; Slavec, A., Berzelak, N., Vehovar, V.
- Web Survey Process within the Concept of eSocial Sciences; 2011; Vehovar, V.
- The Structural Changes of Social Support Networks in a Decade of the Internet Uptake in Slovenia; 2011; Vehovar, V., Petric, G., Petrovčič, A.
- How to ask about fixed/mobile telephone usage? Impact of question wording and analysis of corresponding...; 2011; Slavec, A., Vehovar, V.
- Optimization of dual frame telephone survey designs; 2011; Slavec, A., Vehovar, V.
- Comments on the paper - Geetha Garib: Perceived diversity among employees; 2011; Vehovar, V.
- Costs Error Optimization for Cell-Landline Dual Frame Surveys.; 2011; Slavec, A., Vehovar, V.
- Nonresponse in Web Surveys ; 2011; Lozar Manfreda, K., Berzelak, N., Vehovar, V.
- Mobile Phones in an Environment of Competing Survey Modes: Applying Metric for Evaluation of Costs and...; 2010; Vehovar, V., Berzelak, N., Lozar Manfreda, K.
- Cost-Error Optimization Model for Mixed-Mode Surveys; 2009; Vehovar, V., Berzelak, N., Lozar Manfreda, K.
- Effectiveness of incentives in mixed-mode systems: An evaluation of errors & costs; 2009; Lozar Manfreda, K., Berzelak, N., Vehovar, V.
- Preference for Mobile Interview Surveys? Interplay of costs, errors and biases; 2009; Vehovar, V., Slavec, A.
- National readership surveys: Moving from probability face-to-face surveys to Internet panels; 2009; Vehovar, V., Slavec, A., Petric, I., Sargac, M.
- Empirical Evaluation of Web Survey Software Tools: Powerful or Friendly?; 2009; Vehovar, V., Berzelak, N., Lozar Manfreda, K., Horvat, T., Debevc, M.
- Optimising survey costs in mixed mode environment; 2009; Vehovar, V., Berzelak, N., Lozar Manfreda, K., Belak, E.
- Nonresponse Bias in Surveys; 2009; Bethlehem, J., Vehovar, V., Stoop, I., Schouten, B., Shlomo, N., Skinner, C., Montaquila, J.
- "Mobile phone surveys in mixed mode environment: Balancing costs and errors"; 2009; Vehovar, V.
- Mobile phone surveys in mixed mode environment; 2009; Vehovar, V.
- Towards a flexible online mixed method design with a feedback loop; 2009; Lobe, B., Vehovar, V.
- Probabilistic methods in surveys and offical statistics; 2008; Vehovar, V., Zaletel, M., Seljak, R.
- Internet surveys; 2008; Lozar Manfreda, K., Vehovar, V.