Web Survey Bibliography
Relevance & Research Question: Survey researchers have been forced—by time and budget constraints—to rely on a slew of sampling methods to estimate population parameters. To be valid, sampling procedures must a) have nearly 100% population coverage b) ensure equal chance of selection, and c) feature reasonable response rates. Convenience samples have long been considered inadequate for serious research but collecting opinions from probability samples now takes longer than conveniently interviewing millions of people.
Increasingly, RDD telephone sampling now features response rates below 20%. Moreover, the decrease in landline telephony and the corresponding rise in mobile have put pressure on the claim of near total coverage. Though all online interviewing relies on convenience sampling, it is arguable that people only answer their phones to take surveys (that are a part of RDD studies) when it is convenient for them—a group that continues to shrink.
Methods & Data: This paper investigates whether volume-based approaches to collecting opinions are reasonable substitutes for probability sampling. To do so, we exposed Gallup’s long-used, well documented, United States Presidential approval rating question to a random subset of people who recently completed a survey for one of SurveyMonkey’s seven million survey creators. We presented the question to roughly 10,000 survey takers per day from June 10, 2010 to July 29, 2010. In total, 87, 308 people answered the question.
Results: The results track Gallup’s daily approval numbers within the reported margin of error nearly each day that the test ran without mimicking Gallup’s use of statistical weighting. A closer look reveals that coverage and response rate were respectable relative to those of probability studies. Specifically, respondents were from 8,300 of 19,000 American cities (43%) which approaches the proportion of U.S. households with a landline telephone (60%). Moreover, the respondents yielded an average daily response rate of 46%--more than double that of a typical telephone survey.
Added Value: These findings shed light on recent interest by market researchers in gathering data from technology and social networking sources that have access to extremely large and diverse pools of people and can ask questions of them for virtually no cost.
Conference Homepage (abstract) / (presentation)
Web survey bibliography (4086)
- Experiences with the online questionnaire on www.csfd.cz as a part of mixed research; 2011; Kratka, J.
- Administered versus self-administered online surveys: Are the findings comparable?; 2011; de Rada, V. D.
- Impact of Length or Relevance of Questionnaires on Attrition in Online Trials: Randomized Controlled...; 2011; McCambridge, J., Kalaitzaki, E., White, I. R., Khadjesari, Z., Murray, E., Linke, S., Thompson, S. G...
- Questions for Surveys: Current Trends and Future Directions; 2011; Schaeffer, N. C., Schaeffer, N. C.
- The Future of Modes of Data Collection; 2011; Couper, M. P.
- The Future of Survey Sampling; 2011; Brick, J. M.
- Methodological Issues in Internet-Mediated Research: A Randomized Comparison of Internet Versus Mailed...; 2011; Whitehead, L.
- Why do you play? The development of the motives for online gaming questionnaire (MOGQ); 2011; Demetrovics, Z., Urbán, R., Nagygyörgy, K., Farkas, J., Zilahy, D., Mervó, B., Reindl, A., Agoston,...
- Citizen Satisfaction Survey Data: A Mode Comparison of the Derived-Importance Performance Approach; 2011; Herian, M. N., Tomkins, A. J.
- Surveying ourselves: examining the use of a web-based approach for a physician survey.; 2011; Matteson, K., Anderson, B. L., Pinto, S. B., Clark, M. A., Lopes, V., Schulkin, J.
- Using Online Surveys to Conduct Jury Research; 2011; Edelman, B.
- The Impact of Non-Response Treatments on the Stability of Blockmodels; 2011; Znidarsic, A., Ferligoj, A., Doreian, P.
- The “Frequency Divide” on Web Surveys: Opinions, Behaviours and Demographics Differences...; 2011; Reis, E., Vicente, P.
- LinkedIn and Facebook in Belgium: The Influences and Biases of Social Network Sites in Recruitment and...; 2011; Caers, R., Castelyns, V.
- Apples and oranges: does a web survey produce similar results to social media tracking?; 2011; Bourque, C., Hobbs, R., Hilaire, D. S.
- PhD Students’ Research Group Networks. A Qualitative Approach; 2011; Coromina, L., Capo, A., Coenders, G., Guia, J.
- The influence of Social Support and Personal Networks on Doctoral Student Performance; 2011; Hlebec, V., Kogovsek, T., Ferligoj, A.
- From Busy Bees to Science Geeks and Party Animals: A Typology of Slovenian Doctoral Students; 2011; Hlebec, V., Kogovsek, T., Ferligoj, A.
- PhD Students’ Research Group Social Capital in Two Countries: A Clustering Approach with Duocentred...; 2011; Coenders, G., Coromina, L., Ferligoj, A., Guia, J.
- Preface; 2011; Coenders, G., Doreian, P.
- Does the Inclusion of a Cost Attribute in Forced and Unforced Choices Matter? Results from a Web Survey...; 2011; Pedersen, L. B., Kjaer, T., Kragstrup, J., Gyrd-Hansen, D.
- Explaining Unit Nonresponse in Online Panel Surveys: An Application of the Extended Theory of Planned...; 2011; Haunberger, S.
- Reliability and Validity of Self-Reported Smoking in an Anonymous Online Survey With Young Adults; 2011; Ramo, D. E., Hall, S. M., Prochaska, J. J.
- Nonparametric Tests of Panel Conditioning and Attrition Bias in Panel Surveys; 2011; Das, M., Toepoel, V., van Soest, A.
- Preferences, intentions, and expectation violations: A large-scale experiment with a representative...; 2011; Bellemare. C., Kroeger, S., van Soest, A.
- Twitter mood predicts the stock market.; 2011; Bollen, J., Mao, H., Zeng, X.-J.
- Using Facebook to Locate Sample Members; 2011; Rhodes, B. B., Marks, E. L.
- Paradata in Survey Research; 2011; West, B. T.
- The Perils of Online Surveys; 2011; McCullough, P. R.
- A National Profile of Teacher Education Faculty: The Construction of an Online Survey; 2011; Graber, K. C., Erwin, H., Woods, A. M., Rhoades, J., Zhu, W.
- Validation of the web-based LUMINA questionnaire for recruiting large cohorts of migraineurs; 2011; van Oosterhout, W. P. J., Weller, C. M., Stam, A. H., Stijnen, T., Ferrari, M. D., Terwindt, G. M.
- Exploring Health-related Experiences and Access to Care: Differences between Online and Telephone Survey...; 2011; Doty, M. M., Peugh, J., Shand-Lubbers, J.
- The Impact of “Forgiving” Introductions on the Reporting of Sensitive Behavior in Surveys...; 2011; Peter, J., Valkenburg, P. M.
- Surveying the General Public over the Internet Using Address-Based Sampling and Mail Contact Procedures...; 2011; Messer, B. L., Dillman, D. A.
- Mobile phones as an extension of the participant observer's self: Reflections on the emergent role...; 2011; Hein, W., O'Donohoe, S., Ryan, A.
- Mixed methods designs in marketing research; 2011; Harrison, R. L., Reilly, T. M.
- Introduction to Usability Testing for Survey Research; 2011; Geisen, E., Jarrett, C.
- Utilizing Web Technology in Business Data Collection: Some Norwegian, Dutch and Danish Experiences; 2011; Haraldsen, G., Snijkers, G., Roos, M., Sundvoll, A., Vik, T., Stax, H.-P.
- E-Census 2011 Portugal: implementation and results of the Pilot Survey; 2011; Vicente, P., Rosa, A., Reis, E.
- Facebook sampling methods: some methodological proposals; 2011; Macrì, E., Tessitore, C.
- Reflections on web based data collection in a mixed mode design: the case of the EU Labour Force Survey...; 2011; Kloek, W., van der Valk, J.
- Standardising the web data collection channel at the Basque Statistics Office (EUSTAT); 2011; Prado, C., Guinea , C.
- An Experimental Investigation of Mode Effects in the Hungarian Census Test 2009; 2011; Vereczkei, Z.
- Collaborative systems for enhancing the analysis of social surveys: the Grid Enabled Specialist Data...; 2011; Lambert, P., Warner, G., Doherty, T., McCafferty, S., Watt, J., Comerford, M., Gayle, V., Tan, L., Blum...
- ILS Online Survey; 2011; Weber, C.
- Development of a Web-Based Survey for Monitoring Daily Health and its Application in an Epidemiological...; 2011; Sugiura, H., Ohkusa, Y., Akahane, M., Sano, T., Okabe, N., Imamura, T.
- Sampling v. Scale: An investigation the tension between convenience sampling, response rates, probability...; 2011; Garland, P.
- Effectiveness and consequences of various recruitment methods in psychological research: case study; 2011; Poltorak, M.
- A new approach to the analysis of survey drop-out. Results from Follow-up Surveys in the German Longitudinal...; 2011; Rossmann, J., Blumenstiel, J. E., Steinbrecher, M.
- Tracking the decision-making process – Findings from an Online Rolling Cross-Section Panel Study...; 2011; Faas, T.