Web Survey Bibliography
(a) Relevance & Research Question:
Social scientists are often reluctant to rely on data from online access panels or other web surveys because they fear that the general data quality may be seriously flawed and the results are thus neither valid nor reliable . The third major concern of representative sampling is not always important, e.g. in case of experimental designs or studies that are only used for analyzing relationships as opposed to comparing distributions and mean values. Our research question is, whether speeders in online surveys jeopardize results due to a lack of validity and reliability of results.
(b) Methods & Data:
In a research project on basic human values, fielded in 2010, we have implemented an online questionnaire and gathered data using the ‘Unipark’ online access panel. The 20 minute instrument includes among other measurements the 40 item Schwartz Portrait Value Questionnaire (PVQ) over five consecutive pages. We use confirmatory factor analyses (CFA) and structural equation models (SEM) to analyze our data.
When analyzing the data we hardly faced any item non-response but have discovered many comparatively fast respondents using different speed thresholds. In order to cross-check our results we compared the results of CFA and SEM from two separate surveys (each: n > 750) using independent samples and we compare models from respondents at different speeds. We found a high degree of homogeneity between the two different samples and the models of slow respondents and speeders while controlling for straightliners on our Likert scale.
(d) Added Value:
The results indicate that value research may profit from online access panels both to pretest complex instruments and to validate results, e.g. by applying structural equation models. Although the quality of representative samples is not yet given for most online panels, the tentative results of our fundamental research indicates that speeders do not necessarily jeopardize the quality of the data and thus the validity and the reliability of results. This is an important insight for those social scientists usually relying on cost-intensive traditional PAPI and CATI data collection.
Conference Homepage (abstract) / (presentation)
Web Survey Bibliography - Solicitations & incentives (602)
- Effects of Lotteries on Response Behavior in Online Panels; 2013; Goeritz, A., Luthe, S. C.
- Lotteries and study results in market research online panels; 2013; Goeritz, A., Luthe, S. C.
- Using Web Surveys for Psychology Experiments: A Case Study in New Media Technology for Research; 2013; Peden, B. F., Tiry , A. M.
- Sampling, Channels, and Contact Strategies in Internet Survey; 2013; Macrì, E., Tessitore, C.
- Advancing Research Methods with New Technologies; 2013; Sappleton, N.
- Using mobile devices to access the realities of youth: How identification with society influences political...; 2013; Smith, M.
- Do I Have Your Full Attention?; 2013; Cape, P. J.
- Internet-Based Recruitment to a Depression Prevention Intervention: Lessons From the Mood Memos Study...; 2013; Morgan, A. J., Jorm, A. F., Mackinnon, A. J.
- Challenges for Researchers Investigating Contraceptive Use and Pregnancy Intentions of Young Women Living...; 2013; Herbert, D. L., Loxton, D., Bateson, D., Weisberg, E., Lucke, J. C.
- Pros and cons of virtual interviewers – vote in the discussion about surveytainment; 2013; Póltorak, M., Kowalski, J.
- An Assessment of Incentive Versus Survey Length Trade-offs in a Web Survey of Radiologists; 2013; Ziegenfuss, J. Y., Niederhauser, B. D., Kallmes, D., Beebe, T. J.
- The fish model: What factors affect participants while filling in an online questionnaire?; 2013; Mohamed, B., Lorenz, A., Pscheida, D.
- The monetary value of good questionnaire design; 2013; Tress, F.
- Why are you leaving me?? - Personality predictors of answering drop out in an online-study; 2013; Thielsch, M., Nestler, S., Back, M.
- Seducing the respondent – how to optimise invitations in on-site online research?; 2013; Póltorak, M., Kowalski, J.
- E-questionnaire in cross-sectional household surveys; 2013; Karaganis, M.
- Rewards - Money for Nothing?; 2013; Cape, P. J., Martin, P.
- Effects of incentive reduction after a series of higher incentive waves in a probability-based online...; 2013; Struminskaya, B., Kaczmirek, L., Schaurer, I., Bandilla, W.
- Timing of Nonparticipation in an Online Panel: The effect of incentive strategies; 2013; Douhou, S., Scherpenzeel, A.
- Online Research, Game On!; 2013; Puleston, J.
- Examining the Gender Effects of Different Incentive Amounts in a Web Survey; 2013; Boulianne, S. J.
- How Do Lotteries and Study Results Influence Response Behavior in Online Panels?; 2013; Goeritz, A., Luthe, S. C.
- Incentives for college student participation in web-based substance use surveys; 2013; Patrick, M. E., Singer, E., Boyd, C. J., Cranford, J. A., McCabe, S. E.
- Increasing Web Survey Response Rates in Innovation Research: An Experimental Study of Static and Dynamic...; 2013; Sauermann, H.; Roach, M.
- The Impact of Academic Sponsorship on Online Survey Dropout Rates; 2012; Allen, P. J., Roberts, L. D.
- The Use and Effects of Incentives in Surveys; 2012; Singer, E.
- An Empirical Investigation of the Role of the Email Contact in Web Survey Response Rates; 2012; Hsu, H.-Y., Lai, Y.-H., Chin, H.-Y.
- Mode and non-response effects and their treatment; 2012; Chrysanthopoulos, S., Georgostathi, A.
- The role of topic interest and topic salience in online panel web surveys.; 2012; Keusch, F.
- Effects of E-Mailed Versus Mailed Invitations and Incentives on Response Rates, Data Quality, and Costs...; 2012; Dykema, J., Stevenson, J., Klein, L., Kim, Y., Day, B.
- Prenotification, Incentives, and Survey Modality: An Experimental Test of Methods to Increase Survey...; 2012; Tepper, J. R., Jacob, B.
- Effects of Incentives in Surveys; 2012; Toepoel, V.
- Response rates in three opinion surveys performed through online questionnaires in the health setting...; 2012; Aerny Perreten, N., Domínguez-Berjón, M. F., Astray Mochales, J., Esteban-Vasallo, M. D., Blanco Ancos...
- Detecting Satisficing In Online Surveys: What we found ; 2012; Salifu, S.
- Efficiency of Different Recruitment Strategies for Web Panels; 2012; Hansen, K. M., Pedersen, R. T.
- The Mode of Invitation for Web Surveys; 2012; Bandilla, W., Couper, M. P., Kaczmirek, L.
- Recruiting A Probability Sample For An Online Panel: Effects Of Contact Mode, Incentives, And Information...; 2012; Scherpenzeel, A., Toepoel, V.
- Understanding Society Innovation Panel Wave 4: Results from Methodological Experiments; 2012; Burton, J., Budd, S., Gilbert, E., Jaeckle, A., Kaminska, O., Uhrig, S.C. N., Brown, M., Calderwood,...
- The Propensity of Older Respondents to Participate in a General Purpose Survey; 2012; Lynn, P.
- When More Gets You Less: A Meta-Analysis of the Effect of Concurrent Web Options on Mail Survey Response...; 2012; Medway, R., Fulton, J.
- Does Giving People Their Preferred Survey Mode Actually Increase Survey Participation Rates?; 2012; Olson, K., Smyth, J. D., Wood, H.
- Online Data Collection in the Agro-Food Sector; 2012; Biffignandi, S., Artaz, R.
- Predicting potential respondents' decision to participate in web surveys; 2012; Fang, J., Wen, C.
- Case study: Respondent perspective on survey response; 2012; Jarrett, C.
- Effect of different stimulus on data quality in online panels; 2012; Zagar, S., Lozar Manfreda, K.
- Using the Web to Snowball Discussants of Survey Respondents; 2012; Hopmann, D. N.
- Don't rush through the qualitative screening process ; 2012; Dodson, D.
- The Effectiveness of Survey Recruitment Methods in Second Life; 2012; Dean, E., Cook, S., Murphy, J., Keating, M.
- Improving RDD Cell Phone Samples. Evaluation of Different Pre-call Validation Methods; 2012; Kunz, T., Fuchs, M.
- Influencing Mode Choice in a Mixed Mode Survey; 2012; Mooney, G., Lan, F., Lin, X., Hurwitz, A.