Web Survey Bibliography
In online panels it is common practice to offer incentives for study participation. The current study aimed at answering the following questions: First, how effective are different incentives in enhancing participation in web-based studies? Second, how do these incentives affect data quality? And third, how do these effects change over time?
3227 newly recruited members of an online access panel were invited to seven subsequent survey waves. One half of the participants were offered a result summary for each wave completed, the other half did not receive this offer. In addition, one out of five incentives was promised for participation in each wave. Participants were offered either no incentive, money via bank transfer, money via PayPal, loyalty points, or a donation to charity (either a predetermined charity or a selectable charity). Panelists' willingness to participate was assessed by three measures per wave: Panel attrition, response rate, and retention rate. Data quality was measured in each wave by the number of missing values, the number of words written in answers to open-ended questions, and the tendency to use mid-point categories in close-ended questions. Latent growth curve models and path models were used to analyze the longitudinal effect of the independent variables.
Independent of the manipulations, response rate decreased linearly over the seven waves. Result summary without concurrent incentives accelerated this decline. Adding an incentive to the result summary offset this negative effect, but incentives without concurrent result summary did not heighten the response rate. Donation incentives and PayPal lowered the initial response rate. The number of written words increased by a gentle quadratic slope independent of the manipulations. Either type of incentive increased this quadratic slope. Otherwise, attrition, retention, missing values, and mid-point category use were influenced by incentives merely in a sporadic manner.
The findings suggest that the use of incentives needs to be considered carefully. Panelists might perceive result summaries, Paypal and donations as a burden rather than as gratification. Straightforward payments such as bank transfer and loyalty points appear to perform better, but they need time to bring forth positive effects on panelists' willingness to participate.
Conference Homepage (abstract) / (presentation)
Web survey bibliography (290)
- Mind the Mode: Differences in Paper vs. Web-Based Survey Modes Among Women With Cancer; 2017; Hagan, T. L.; Belcher, S. M.; Donovan, H. S.
- Telephone versus Online Survey Modes for Election Studies: Comparing Canadian Public Opinion and Vote...; 2017; Breton, C.; Cutler, F.; Lachance, S.; Mierke-Zatwarnicki, A.
- Paradata as an aide to questionnaire design: Improving quality and reducing burden; 2017; Timm, E.; Stewart, J.; Sidney, I.
- Fieldwork monitoring and managing with time-related paradata; 2017; Vandenplas, C.
- Where, When, How and with What Do Panel Interviews Take Place and Is the Quality of Answers Affected...; 2017; Niebruegge, S.
- Do distractions during web survey completion affect data quality? Findings from a laboratory experiment...; 2017; Wenz, A.
- A Comparison of Emerging Pretesting Methods for Evaluating “Modern” Surveys; 2017; Geisen, E., Murphy, J.
- Pushing to web in the ISSP; 2017; Jonsdottir, G. A.; Dofradottir, A. G.; Einarsson, H. B.
- Push2web or less is more? Experimental evidence from a mixed-mode population survey at the community...; 2017; Neumann, R.; Haeder, M.; Brust, O.; Dittrich, E.; von Hermanni, H.
- Rates, Delays, and Completeness of General Practitioners’ Responses to a Postal Versus Web-Based...; 2017; Sebo, P.; Maisonneuve, H.; Cerutti, B.; Pascal Fournier, J.; Haller, D. M.
- Comparing data quality and cost from three modes of on-board transit surveys ; 2017; Agrawal, A. W.; Granger-Bevan, S.; W.; Newmark, G. L.; Nixon, H.
- Overview: Online Surveys; 2017; Vehovar, V.; Lozar Manfreda, K.
- FocusVision 2015 Annual MR Technology Report; 2016; Macer, T., Wilson, S.
- Comparisons of Online Recruitment Strategies for Convenience Samples: Craigslist, Google AdWords, Facebook...; 2016; Antoun, C., Zhang, C., Conrad, F. G., Schober, M. F.
- Feature phones no barrier to conducting an effective conjoint study ; 2016; de Rooij, R.; Dossin, R.
- Mixing Modes: Challenges (and Tradeoffs) of Adapting a Mailed Paper Survey to the Web ; 2016; Wilkinson-Flicker, S.; McPhee, C. B.; Medway, R.; Kaiser, A.; Cutts, K.
- Does Changing Monetary Incentive Schemes in Panel Studies Affect Cooperation? A Quasi-experiment on...; 2016; Schaurer, I.; Bosnjak, M.
- Survey Mode and Mail Method: A Practical Experiment in Survey Fielding for a Multi-round Survey ; 2016; Sullivan, B. D.; Duda, N.; Bogen, K.; Clusen, N. A.; Wakar, B.; Zhou, H.
- Early-bird Incentives: Results From an Experiment to Determine Response Rate and Cost Effects ; 2016; De Santis, J.; Callahan, R.; Marsh, S.; Perez-Johnson, I.
- Using Cash Incentives to Help Recruitment in a Probability Based Web Panel: The Effects on Sign Up Rates...; 2016; Krieger, U.
- Assessing Changes in Coverage Bias of Web Surveys a s Internet Access Increases in the United States...; 2016; Sterrett, D.; Malato, D.; Benz, J.; Tompson, T.; English, N.
- Thinking Inside the Box Visual Design of the Response Box Affects Creative Divergent Thinking in an...; 2016; Mohr, A. H.; Sell, A.; Lindsay, T.
- Web surveys for offline rural communities ; 2016; Gichohi, B. W.
- Adaptive survey designs to minimize survey mode effects – a case study on the Dutch Labor Force...; 2016; Calinescu, M.; Schouten, B.
- Assessment of Innovations in Data Collection Technology for Understanding Society; 2016; Couper, M. P.
- Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk; 2016; Berinsky, A.; Huber, G. A.; Lenz, G. S.
- Sample Representation and Substantive Outcomes Using Web With and Without Incentives Compared to Telephone...; 2016; Lipps, O.; Pekari, N.
- “Money Will Solve the Problem”: Testing the Effectiveness of Conditional Incentives for...; 2016; DeCamp, W.; Manierre, M. J.
- Effects of Incentive Amount and Type of Web Survey Response Rates; 2016; Coopersmith, J.; Vogel, L. K.; Bruursema, T.; Feeney, K.
- Effect of a Post-paid Incentive on Response to a Web-based Survey; 2016; Brown, J. A.; Serrato, C. A.; Hugh, M.; Kanter, M. H.; A.; Spritzer, K. L.; Hays, R. D.
- Take the money and run? Redemption of a gift card incentive in a clinician survey. ; 2016; Chen, J. S.; Sprague, B. L.; Klabunde, C. N.; Tosteson, A. N. A.; Bitton, A.; Onega, T.; MacLean, C....
- A reliability analysis of Mechanical Turk data; 2016; Rouse, S. V.
- Taming Big Data: Using App Technology to Study Organizational Behavior on Social Media; 2015; Bail, C. A.
- Using Mobile Phones for High-Frequency Data Collection; 2015; Azevedo, J. P.; Ballivian, A.; Durbin, W.
- Cell Phone and Face-to-face Interview Responses in Population-based Sur- veys - How Do They Compare?; 2015; Ghandour, L.; Ghandour, B.; Mahfoud, Z.; Mokdad, A.; Sibai, A. M.
- The Cathie Marsh lecture: What does the failure of the polls tell us about the future of survey research...; 2015; Sturgis, P., Matheson, J.
- Mixed mode surveys ; 2015; Burton, J.
- Understanding Society Innovation Panel Wave 7: Results from Methodological Experiments; 2015; Blom, A. G.; Burton, J.; Booker, C. L.; Cernat, A.; Fairbrother, M.; Jaeckle, A.; Kaminska, O.; Keusch...
- Response Effects of Prenotification, Prepaid Cash, Prepaid Vouchers, and Postpaid Vouchers: An Experimental...; 2015; van Veen, F.; Goeritz, A.; Sattler, S.
- Tailored fieldwork design to increase representative household survey response: an experiment in the...; 2015; Luiten, A.; Schouten, B.
- Using Internet to Recruit Immigrants with Language and Culture Barriers for Tobacco and Alcohol Use...; 2015; Carlini, B. H.; Safioti, L.; Rue, T. C.; Miles, L.
- Online Recruitment Methods for Web-Based and Mobile Health Studies: A Review of the Literature; 2015; Lane, T. S.; Armin, J.; Gordon, Ju. S.
- iTunes Song-Gifting is a Low-Cost, Efficient Recruitment Tool to Engage High-Risk MSM in Internet Research...; 2015; Holland, C. M.; Ritchie, N. D.; Du Bois, S. N.
- A Note on the Use of Survey Research Firms to Enable Empirical Data Collection; 2015; Schoenherr, T.; Ellram, L. M.; Tate, W. L.
- Recruiting Online: Lessons From a Longitudinal Survey of Contraception and Pregnancy Intentions of Young...; 2015; Harris, M. L.; Loxton, D.; Wigginton, B.; Lucke, J. C.
- Recruiting for addiction research via Facebook; 2015; Thornton, L. K.; Harris, K.; Baker, A.; Johnson, M.; Kay-Lambkin, F. J.
- The effectiveness of incentives on recruitment and retention rates: an experiment in a web survey; 2015; Mulder, J.; Douhou, S.
- Adding a Web Mode to Phone Surveys: Effectiveness and Cost Implications; 2015; Beebe, T. J.; Lien, R.; Luxenberg, H.; Rainey, J.
- Query on Data Collection for Social Surveys; 2014; Blanke, K., Luiten, A.
- Examining the Effect of Prenotification Postcards on Online Survey Response Rate in a University Graduate...; 2014; Lalasz, C. B., Doane, M. J., Springer, V. A., Dahir, V. B.