Web Survey Bibliography
(a) Relevance & Research Question:
Social scientists are often reluctant to rely on data from online access panels or other web surveys because they fear that the general data quality may be seriously flawed and the results are thus neither valid nor reliable . The third major concern of representative sampling is not always important, e.g. in case of experimental designs or studies that are only used for analyzing relationships as opposed to comparing distributions and mean values. Our research question is, whether speeders in online surveys jeopardize results due to a lack of validity and reliability of results.
(b) Methods & Data:
In a research project on basic human values, fielded in 2010, we have implemented an online questionnaire and gathered data using the ‘Unipark’ online access panel. The 20 minute instrument includes among other measurements the 40 item Schwartz Portrait Value Questionnaire (PVQ) over five consecutive pages. We use confirmatory factor analyses (CFA) and structural equation models (SEM) to analyze our data.
When analyzing the data we hardly faced any item non-response but have discovered many comparatively fast respondents using different speed thresholds. In order to cross-check our results we compared the results of CFA and SEM from two separate surveys (each: n > 750) using independent samples and we compare models from respondents at different speeds. We found a high degree of homogeneity between the two different samples and the models of slow respondents and speeders while controlling for straightliners on our Likert scale.
(d) Added Value:
The results indicate that value research may profit from online access panels both to pretest complex instruments and to validate results, e.g. by applying structural equation models. Although the quality of representative samples is not yet given for most online panels, the tentative results of our fundamental research indicates that speeders do not necessarily jeopardize the quality of the data and thus the validity and the reliability of results. This is an important insight for those social scientists usually relying on cost-intensive traditional PAPI and CATI data collection.
Conference Homepage (abstract) / (presentation)
Web Survey Bibliography - 2011 (566)
- A Methodological Inference towards the Quantification of Technological Frames ; 2011; Cachia, E., Camilleri, P.
- International Economics, Labour markets and Web Surveys Qualities; 2011; Guzi, M., de Pedraza, P.
- On Affordances and Technological Intersubjectivity; 2011; Vatrapu, R.
- The optimal number of response options in internet panel surveys; 2011; Thorsdottir, F.
- Online Research @ GESIS; 2011; Kaczmirek, L., Lenzner, T.
- Building online communities; 2011; Mlačić, B., Milas, G., Mikloušić, I.
- Romanian financing opportunities for the action; 2011; Moga, L.
- Current Projects at University of Ljubljana; 2011; Lozar Manfreda, K.
- 2nd WEBDATANET Meeting Amsterdam. Global Data Collecting; 2011; van Beveren, I.
- German Web-based Registry for Eating Disorders; 2011; Gross, G., Birgegård, A., Zipfel, S.
- Collecting information with Knowledge technologies; 2011; Foulonneau, M.
- E-dater, Artificial Actors, and German Households; 2011; Hebing, M.
- DIME-SHS; 2011; Lesnard, L.
- Web based Data Collection – A Jungle becoming a Field or a Field becoming a Jungle? ; 2011; Ole Finnemann, N.
- Online Surveys: Past, Present, and Future; 2011; de Leeuw, E. D.
- Vademecum: COST instruments, Core Group Meeting; 2011
- Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys 2011; 2011
- Calibrating Non-Probability Internet Samples with Probability Samples Using Early Adopter Characteristics...; 2011; DiSogra, C., Cobb, C. L., Chan, E., Dennis, J. M.
- A randomised trial and economic evaluation of the effect of response mode on response rate, response...; 2011; Scott, A., Jeon, S.-H., Joyce, C. M., Humphreys, J. S., Kalb, G., Witt, J., Leahy, A.
- Web Survey Methodology: Interface Design, Sampling and Statistical Inference; 2011; Couper, M. P.
- Globalpark Annual Market Research Software Survey 2010; 2011; Macer, T.; Wilson, Sheila
- Effect of interview modes on measurement of identity; 2011; Nandi, A., Platt, L.
- Maintaining Cross-Sectional Representativeness in a Longitudinal General Population Survey ; 2011; Lynn, P.
- Understanding Society Innovation Panel Wave 3: Results from Methodological Experiments; 2011; Burton, J., Budd, S., Gilbert, E., Jaeckle, A., McFall, S., Uhrig, S.C. N.
- The Effect of a Mixed Mode Wave on Subsequent Attrition in a Panel Survey: Evidence from the Understanding...; 2011; Lynn, P.
- Seeing Through the Eyes of the Respondent: An Eye-tracking Study on Survey Question Comprehension; 2011; Lenzner, A., Kaczmirek, L., Galesic, M.
- Eye Tracking in testing questionnaires: What’s the added value?; 2011; Tries, S.
- Panel Recruitment via Facebook; 2011; Toepoel, V.
- Dropout rates during completion of an occupation search tree in web-surveys; 2011; Tijdens, K.
- Usability and burden measurement in online forms; 2011; Thomsen, P.
- Dynamic Data Editing in online data collection for the Vacant Positions Survey; 2011; Stax, H.-P.
- Utilizing Web Technology in Business Data Collection: Some Norwegian, Dutch and Danish Experiences; 2011; Snijkers, G., Haraldsen, G., Stax, H.-P.
- Web survey software; 2011; Slavec, A., Berzelak, N., Vehovar, V.
- Disentangling relative mode effects for the web survey mode in the Safety Monitor; 2011; Schouten, B., van de Brakel, J., Buelens, B., Klausch, L. T., van der Laan, J.
- Combining web surveys and XBRL ; 2011; Roos, M.
- Improving validity in web surveys with hard‐to‐reach targets: Online Respondent Driven Sampling...; 2011; Mavletova, A. M.
- Developing Electronic Questionnaires at Statistics Canada: Experiences and Challenges in a Changing...; 2011; Lawrence, D.
- Experiences with mixed mode mail & web-enquêtes in probability samples with known individuals; 2011; Kalgraff Skjak, K., Kolsrud, K.
- Towards a Holistic Questionnaire Develoment, Evaluation and Testing Tool; 2011; Haraldsen, G., Revold, M. K.
- Effects of internet data collection in business surveys – the case of the Dutch SBS; 2011; Giesen, D.
- Ignoring the compatibility of online questionnaires may bias the psychological composition of your sample...; 2011; Funke, F.
- Video enhanced web survey; 2011; Fuchs, M., Kunz, T., Gebhard, F.
- Keeping Up Appearances: Maintaining standards during strategic changes in electronic reporting; 2011; Farrell, E., Hewett, K.
- Respondent engagement: using usability testing; 2011; Dowling, Z.
- Yes, no, maybe. Effectiveness and Effects of Recruiting Strategies for Online Access Panels; 2011; Dale, T., Marksten, O. M. Walsoe, H.
- New data sources for statistics: Experiences at Statistics Netherlands; 2011; Daas, P., ten Bosch, O., de Blois, C., Hoekstra, R., Roos, M.
- Further research on the design of complex grids; 2011; Couper, M. P., Tourangeau, R.Conrad, F. G., Zhang, C.
- Interactive carrots and sticks to improve data quality; 2011; Conrad, F. G., Tourangeau, R., Couper, M. P., Zhang, C.
- Scrolling or paging - it depends; 2011; Blanke, K.
- Mode of invitation for web surveys; 2011; Bandilla, W., Couper, M. P.