Web Survey Bibliography
(a) Relevance & Research Question:
Social scientists are often reluctant to rely on data from online access panels or other web surveys because they fear that the general data quality may be seriously flawed and the results are thus neither valid nor reliable . The third major concern of representative sampling is not always important, e.g. in case of experimental designs or studies that are only used for analyzing relationships as opposed to comparing distributions and mean values. Our research question is, whether speeders in online surveys jeopardize results due to a lack of validity and reliability of results.
(b) Methods & Data:
In a research project on basic human values, fielded in 2010, we have implemented an online questionnaire and gathered data using the ‘Unipark’ online access panel. The 20 minute instrument includes among other measurements the 40 item Schwartz Portrait Value Questionnaire (PVQ) over five consecutive pages. We use confirmatory factor analyses (CFA) and structural equation models (SEM) to analyze our data.
When analyzing the data we hardly faced any item non-response but have discovered many comparatively fast respondents using different speed thresholds. In order to cross-check our results we compared the results of CFA and SEM from two separate surveys (each: n > 750) using independent samples and we compare models from respondents at different speeds. We found a high degree of homogeneity between the two different samples and the models of slow respondents and speeders while controlling for straightliners on our Likert scale.
(d) Added Value:
The results indicate that value research may profit from online access panels both to pretest complex instruments and to validate results, e.g. by applying structural equation models. Although the quality of representative samples is not yet given for most online panels, the tentative results of our fundamental research indicates that speeders do not necessarily jeopardize the quality of the data and thus the validity and the reliability of results. This is an important insight for those social scientists usually relying on cost-intensive traditional PAPI and CATI data collection.
Conference Homepage (abstract) / (presentation)
Web Survey Bibliography (6359)
- Using screen video capture software to aide and inform cognitive interviewing; 2012; Chaney, B. H., Barry, A. E., Chaney, J. D., Stellefson, M. L., Webb, M. C.
- Respondents Cooperation: Demographic Profile of Survey Respondents and Its Implication; 2012; Glaser, P.
- Costs and Errors in Fixed and Mobile Phone Surveys; 2012; Vehovar, V., Slavec, A., Berzelak, N.
- E-Mail Surveys; 2012; Mesch, G.
- Does survey experience affect respondents’ reported level of satisfaction?; 2012; Schultz Christensen, A., Ladenburg, J.
- Building Your Own Online Panel Via E-Mail and Other Digital Media; 2012; Toepoel, V.
- Data Quality in HIV/AIDS Web-Based Surveys: Handling Invalid and Suspicious Data; 2012; Bauermeister, J. A., Pingel, E., Zimmerman, M., Couper, M. P., Carballo-Diéguez, A., Strecher, V. J.
- Response rates in three opinion surveys performed through online questionnaires in the health setting...; 2012; Aerny Perreten, N., Domínguez-Berjón, M. F., Astray Mochales, J., Esteban-Vasallo, M. D., Blanco Ancos...
- Use of Web 2.0 to Recruit Australian Gay Men to an Online HIV/AIDS Survey; 2012; Theriault, N., Bi, P., Hiller, J. E., Nor, M.
- Web and Mail Surveys: An Experimental Comparison of Methods for Nonprofit Research; 2012; Lin, W., Van Ryzin, G. G.
- Evaluation of an online (opt-in) panel for public participation geographic information systems surveys...; 2012; Brown, G., Weber, D., Zanon, D., de Bie, K.
- The World's Simplest Survey Microsimulator (WSSM); 2012; Karr, A. F., Cox, L. H.
- Survey Data Collection and Integration; 2012; Davino, C., Fabbris, L.
- Detecting Satisficing In Online Surveys: What we found ; 2012; Salifu, S.
- Developing an Online GIS-based Survey Instrument to Elicit Perceived Neighborhood Information for Environmental...; 2012; Carr, J., Johnson, I., Keene, R., Landry, P., Gradeck, R., Lucht, J., Beach, S., Clougherty, J.
- Challenges of assessing the quality of a prerecruited probability-based panel of internet users in...; 2012; Struminskaya, B., Kaczmirek, L.
- Item comparability in cross-national surveys: results from asking probing questions in cross-national...; 2012; Behr, D., Braun, M., Kaczmirek, L., Bandilla, W.
- Assessing Cross-National Equivalence of Measures of Xenophobia: Evidence from Probing in Web Surveys; 2012; Behr, D., Braun, M., Kaczmirek, L.
- Impact and the Research Excellence Framework: New challenges for universities; 2012; Grant, J.
- Mobile Befragungen: Was Big Data mit kleinen Geräten zu tun hat; 2012
- Studying Migrants with the Help of the Internet: Methods from Psychology; 2012; Reips, U. -D., Buffardi, L.
- Impact of Fixed Choice Design on Blockmodeling Outcomes; 2012; Znidarsic, A.
- Panel Conditioning in Online Survey Panels: Problems of Increased Sophistication and Decreased Engagemeent...; 2012; Adams, A. N., Atkeson, L. R., Karp, J. A.
- Efficiency of Different Recruitment Strategies for Web Panels; 2012; Hansen, K. M., Pedersen, R. T.
- Understanding Mode Effects between Mobile Web and Mobile SMS Surveys; 2012; Poduska, B., Johnson, P.
- Improving Representativeness In Online Surveys Using A Combined Internet/Postal Approach: Evidence From...; 2012; Atorough, P., Donaldson, B., Harris, A.
- Paper-and-Pencil versus Web Administration of a Student Satisfaction Survey; 2012; Bowen, C.-C.
- Toward Understanding Response Sequence in Check-All-That-Apply Web Survey Questions: A Research Note...; 2012; Stern, M. J., Bilgen, I., Rookey, B. D.
- The Mode of Invitation for Web Surveys; 2012; Bandilla, W., Couper, M. P., Kaczmirek, L.
- Encouraging Survey Response via Smartphones: Effects on Respondents’ Use of Mobile Devices and...; 2012; Dillman, D. A., Millar, M. M.
- Surveying Rare Populations Using a Probabilitybased Online Panel; 2012; Peugh, J., Wright, G.
- Methods for Improving Response Rates in Two-Phase Mail Surveys; 2012; Brick, J. M., Andrews, W. R., Brick, P. D., King, H., Mathiowetz, N. A., Stokes, L.
- Latent class analysis of response inconsistencies across modes of data collection; 2012; Yan, T., Kreuter, F., Tourangeau, R.
- Item non-response in open-ended questions: Who does not answer on the meaning of left and right?; 2012; Scholz, E., Zuell, C.
- Nonresponse and Online Student Evaluations of Teaching: Understanding the Inﬂuence of Salience...; 2012; Adams, M. J. D., Umbach, P. D.
- Disfluencies and Gaze Aversion in Unreliable Responses to Survey Questions; 2012; Schober, M. F., Conrad, F. G., Dijkstra, W., Ongena, Y. P.
- Evaluating Survey Questions: A Comparison of Methods; 2012; Yan, T., Kreuter, F., Tourangeau, R.
- Use of Paradata in a Responsive Design Framework to Manage a Field Data Collection; 2012; Wagner, J., West, B. T., Kirgis, N., Lepkowski, J. M., Axinn, W., Kruger-Ndiaye, S.
- A Comparison of Alternative Indicators for the Risk of Nonresponse Bias; 2012; Wagner, J.
- Differences by Survey Language and Mode among Chinese Respondents to a CAHPS Health Plan Survey; 2012; Elliott, M. N., Edwards, W. S., Klein, J. D., Heller, A.
- Recruiting A Probability Sample For An Online Panel: Effects Of Contact Mode, Incentives, And Information...; 2012; Scherpenzeel, A., Toepoel, V.
- Innovation der Online-Datenerhebung für wissenschaftliche Forschungen. Das niederländische MESS-Projekt...; 2012; Das, M.
- Do Questions about Watching Internet Pornography Make People Watch Internet Pornography? A Comparison...; 2012; Peter, J., Valkenburg, P. M.
- “I think I know what you did last summer” Improving data quality in panel surveys; 2012; Lugtig, P. J.
- An Initial Look at Non-Response and Attrition in Understanding Society; 2012; Lynn, P., Burton, J., Kaminska, O., Knies, G., Nandi, A.
- Understanding Society Innovation Panel Wave 4: Results from Methodological Experiments; 2012; Burton, J., Budd, S., Gilbert, E., Jaeckle, A., Kaminska, O., Uhrig, S.C. N., Brown, M., Calderwood,...
- The Propensity of Older Respondents to Participate in a General Purpose Survey; 2012; Lynn, P.
- Mode-Switch Protocols: How a Seemingly Small Design Difference can affect Attrition Rates and Attrition...; 2012; Lynn, P.
- When More Gets You Less: A Meta-Analysis of the Effect of Concurrent Web Options on Mail Survey Response...; 2012; Medway, R., Fulton, J.
- Does Giving People Their Preferred Survey Mode Actually Increase Survey Participation Rates?; 2012; Olson, K., Smyth, J. D., Wood, H.