Web Survey Bibliography
The purpose of this paper is to test whether probability samples and non-probability samples of volunteers yield different survey results. Nine survey companies collected data with an identical questionnaire, one via RDD telephone interviewing, one via the Internet with a sample recruited via RDD telephone interviewing, and seven via the Internet with non-probability samples of volunteers. We first compared the demographic distributions of the nine samples with the Current Population Survey (CPS). Some preliminary results suggest that the distributions of the two probability samples are generally comparable to CPS, while the seven non-probability samples tend to underrepresent minority groups, such as Blacks and Hispanics, lower-educated people, etc. We then compared the two probability samples to assess their accuracy via comparisons with government and industry benchmarks when they are available. Some preliminary finding suggests that the RDD Internet sample produced more accurate results compared to the RDD telephone sample, with regard to only those questions that we have benchmarks to compare to. But for some other questions, the two probability samples generated quite different results. We will further explore whether these differences were due to mode effects or other systematic biases. We also compared the two probability samples to the seven non-probability samples in terms of accuracy. While some of the results were comparable between probability samples and non-probability samples, others were found significantly different. For one non-probability sample, most of the results were not comparable to other companies, which might be attributed to the extremely skewed demographic distribution of the sample. To adjust this, we will examine whether data quality of the non-probability samples would be improved by creating weights to match the survey demographics to the nation. Lastly, we will compare whether experiments embedded in the survey yielded comparable results across the platforms.
Homepage (abstract)
Web survey bibliography - WAPOR 61th Annual Conference, 2008 (55)
- Testing the Effects of Multiple Manipulations on Print and Online Survey Response Rates: Lessons Learned...; 2008; Bachman, M., Vaccaro, D.
- The Role of Cash Incentives in Online Panelist Motivations: Experimental Results on Unit Response and...; 2008; Taylor, E.
- ‘For Example’: How Different Example Types in Online Surveys Influence Frequency ; 2008; Berent, M., Krosnick, J. A.
- Validating Check-All and Forced-Choice Question in a Paper Survey of Provincial Park Campground Users...; 2008; Dyck, B., Moore, D.
- Measuring Attentiveness to Current Events in a Mixed Mode Experiment; 2008; Suls, R., Horowitz, J.
- Transitioning from Self-Reports to Self-Installed Electronic Audience Measurement; 2008; Trussell, N., Vanno, L., Matthess, E., Bailey, J., Link, M. W.
- The Role of New Technology and its’ Effect on Best Practices Methodology; 2008; Kendall, E.
- Rate of Response in Web-Based Data Collection as a Factor of Author of E-mail Invitation; 2008; Mitra, A.
- Graduate vs. Undergraduate Student Respondent Behavior Differences in Web Surveys; 2008; Showen, S., Eisenberg, D., Roe, D. J.
- Mode Effects and Non-Response Bias in an Undergraduate Student Satisfaction Survey: Results from a Randomized...; 2008; Beach, S., Musa, D., Beeson, P., Sparks, C.
- Worth the Weight?: The Benefits and Pitfalls in Applying Survey Weights to Web Surveys of College Undergraduates...; 2008; Bloom, J. D.
- Improving the Efficiency of Web Survey Experiments; 2008; Luks, S., Rivers, D.
- When Encouraging Looks Go Too Far: Using Virtual Humans to Understand the Role of Rapport in the Survey...; 2008; Foucault, B., Aguilar, J., Cassell, J., Miller, P. V.
- Social Cues Can Affect Answers to Threatening Questions in Virtual Interviews; 2008; Lind, L. H., Schober, M. F., Conrad, F. G.
- How Animated Agents Affect Responses in Open-Ended Interviews about Alcohol Use; 2008; Person, N. K.
- Virtual Interviews on Mundane, Non-Sensitive Topics: Dialog Capability Affects Response Accuracy More...; 2008; Conrad, F. G., Schober, M. F., Jans, M., Orlowski, R. A., Nielsen, D.
- Mall-Intercept vs. Online Panel: Does Sample Source for an Experimental Study Matter?; 2008; Lin, C. T. J.
- Representativeness in Online-Surveys Through Stratified Sample; 2008; Blasius, J.
- Comparing the Results of Probability and Non-probability Telephone and Internet Survey Data; 2008; Wang, R., Krosnick, J. A.
- Evaluating the Potential Contributions of a Web-based Convenience Sample to the Accuracy of a Probability...; 2008; Elliott, M. N., Haviland, A.
- “R U in the Network?!” Using Text Messaging Interfaces as Screeners for Working Cell Phone...; 2008; Buskirk, T. D., Rao, K., Callegaro, M., Arens, Z., Steiger, D. M.
- Sampling & Weighting Cell Phone Samples to Supplement RDD Surveys; 2008; Brick, J. M., Edwards, W. S., Lee, Sunghee
- Using the ESRC Question Bank: An Online Resource Developed for the Social Survey Research Community; 2008; Gibbs, J. C.
- Why Text Mine?; 2008; Parry, J., Tomashek, S.
- Internet Access Panels and Public Opinion and Attitude Estimates; 2008; Piekarski, L., Galin, M., Baim, J., Frankel, M. R., Augemberg, K., Prince, S.
- Combining Mail and Internet Methods to Conduct Household Surveys of the General Public: A New Methodology...; 2008; Dillman, D. A., Smyth, J. D., Christian, L. M., Oneill, A.
- Experiment on Use of Internet Cell Phone Only Panelists to Supplement RDD Samples; 2008; Turakhia, C., Schulman, M. A., Bohinsky, S.
- Evaluating Efficiency and Effectiveness of Cell Phone Samples; 2008; Sen, S., Zmud, J., Arce, C.
- Does the Inclusion of Mail and Web Alternatives in a Probability-Based Household Panel Improve the Accuracy...; 2008; Rookey, B. D., Dillman, D. A., Hanway, S.
- The “Professional Respondent” Problem in Web Surveys; 2008; Rivers, D.
- Predictors and Barriers to Collecting Data from Early Childhood Educators Using the Web; 2008; Caspe, M., Sonnenfeld, K., Meagher, C., Sprachman, S., Scaturro, G.
- The MacroPoll Wireless Experience: Development and Lessons Learned.; 2008; Austin, J. D., Zullwack, R., Dyer, A., Dayton, J. J.
- My Cell Phone’s Ringing, “Caller Unknown,” Now What? Usage Behavior Patterns Among...; 2008; Buskirk, T. D., Rao, K., Kaminska, O.
- Pilot Development of a Smartphone-Enabled Full-Probability Panel; 2008; Hill, C., Biemer, P. P., Coombs, D., Eyerman, J.
- A Test of Short versus Long Cell Phone Interviews; 2008; Jones, Je.
- Evaluating the Characteristics of Landline User’s Intention to Switch to Cell Phone Only Use for...; 2008; Sanderson, M., Immerwahr, S., Eisenhower, D., Konty, K.
- Coverage Bias in Surveys Excluding Cell Phone Only Adults: Evaluation of Bias and Effectiveness of Post...; 2008; Peytchev, A., Carley-Baxter, L. R., Black, M. L.
- Landline and Cell Phone Usage Patterns Among Young Adults; 2008; Currivan, D. B., Roe, D. J., Stockdale, J.
- Practical Steps to Conducting Cellular Telephone Surveys; 2008; Howes, C., DeBello, A., Wolter, K., Wooten, K.
- Health Policy Concerns and Policy Preferences: A Comparison of Landline RDD and Cell Phone Only (and...; 2008; Zukin, C., Cantor, J., Brownlee, S., Boyle, J.
- Measuring Health in RDD Surveys: Are Estimates that Exclude the Cell-Only Population Accurate?; 2008; Freedner, N., Holterman, L. A., Hannah, K.
- Does Including Cell-Only Households in an RDD Survey Change the Estimates? The Case of the American...; 2008; Bryant, B. E., Baker, R. P.
- The Effects of Excluding Cell-Only Respondents on Understanding Religion in the United States; 2008; Smith, G. A., Cox, D., Pond, A.
- Use of FedEx: Early, Late or Never?; 2008; Pope, D.
- When is the Best Time to Invite a Respondent? An Analysis of E-mail Invitation Timing and Response to...; 2008; Sinibaldi, J., Hansen, S. E.
- Instant Messaging: Applicability for Contacting Potential Web Respondents?; 2008; Cox, C. J., Harwood, P. G., Swanhart, M.
- E-mail and Postcard Invitation Designs to Maximize Web-Survey Responses Rates; 2008; Kaplowitz, M. D., Lupi, F., Couper, M. P., Thorp, L.
- Latent Class Modeling in Survey Methods: Estimation of the Cell Phone Only Population; 2008; Albaghal, M.
- Calculating Response Rates for Cell Telephone Surveys; 2008; Barron, M., Khare, M.
- Predicting Survey Bias in a Brave New Mobile World: Using the Behavioral Theory of Lifestyle Adoption...; 2008; Ehlen, P., Ehlen, J.