Web Survey Bibliography
(a) Relevance & Research Question:
Social scientists are often reluctant to rely on data from online access panels or other web surveys because they fear that the general data quality may be seriously flawed and the results are thus neither valid nor reliable . The third major concern of representative sampling is not always important, e.g. in case of experimental designs or studies that are only used for analyzing relationships as opposed to comparing distributions and mean values. Our research question is, whether speeders in online surveys jeopardize results due to a lack of validity and reliability of results.
(b) Methods & Data:
In a research project on basic human values, fielded in 2010, we have implemented an online questionnaire and gathered data using the ‘Unipark’ online access panel. The 20 minute instrument includes among other measurements the 40 item Schwartz Portrait Value Questionnaire (PVQ) over five consecutive pages. We use confirmatory factor analyses (CFA) and structural equation models (SEM) to analyze our data.
When analyzing the data we hardly faced any item non-response but have discovered many comparatively fast respondents using different speed thresholds. In order to cross-check our results we compared the results of CFA and SEM from two separate surveys (each: n > 750) using independent samples and we compare models from respondents at different speeds. We found a high degree of homogeneity between the two different samples and the models of slow respondents and speeders while controlling for straightliners on our Likert scale.
(d) Added Value:
The results indicate that value research may profit from online access panels both to pretest complex instruments and to validate results, e.g. by applying structural equation models. Although the quality of representative samples is not yet given for most online panels, the tentative results of our fundamental research indicates that speeders do not necessarily jeopardize the quality of the data and thus the validity and the reliability of results. This is an important insight for those social scientists usually relying on cost-intensive traditional PAPI and CATI data collection.
Conference Homepage (abstract) / (presentation)
Web Survey Bibliography - 2011 (584)
- Background - QSOAP; 2011
- Audience evolution: New technologies and the transformation of media audiences; 2011; Napoli, P. M.
- Assessing personality traits through response latencies using item response theory; 2011; Ranger, J., Ortner, T. M.
- American public opinion: Its origins, content, and impact (8th Edition); 2011; Erikson, R. S., Tedin, K. L.
- Amazon's Mechanical Turk. A new source of inexpensive, yet high quality, data?; 2011; Buhrmester, M., Kwang, T., Gosling, S. D.
- A theory of public opinion. A reprint of a classic book with a new introduction by H. Lee Cheek, Jr.; 2011; Wilson, F.
- Clarifying Survey Questions; 2011; Redline, C. D.
- A new representative standard for online research: Conquering the challenge of the dirty little "...; 2011; Gittelman, S., Trimarchi, E., Fawson, B.
- 2011 Skills for Life Survey: Headline findings ; 2011
- The place for mobile research? Multi-mode studies of major cultural events; 2011; Conry, S., Atkinson, S.
- Facial imaging: The new face of online survey research; 2011; Gordon, A., McCallum, D., Sorci, M., Llewellyn, T.
- How far is too far: Traditional, flash and gamification interfaces, and implications for the future...; 2011; Puleston, J., Malinoff, B.
- The Evolution of Edits in the Canadian Census of Population Online Questionnaires; 2011; Laroche, D.
- The Main Innovations of Data Editing and Imputation for the 2010 Italian Agricultural Census ; 2011; Bianchi, G., Lipsi, R. M., Ruocco, G., Salvatore, M. A.
- A classification of question characteristics relevant to measurement (error) and consequently important...; 2011; Campanelli, P., Nicolaas, G., Jaeckle, A., Lynn, P., Hope, S., Blake, M., Gray, M.
- Hrh remuneration: Comparing wage levels, ranking And dispersion of 16 occupations In the health workforce...; 2011; Tijdens, K., de Vries, D.
- Wages worldwide results and measurement issues from the multi-country. WageIndicator web-survey ; 2011; van Klaveren, M., Tijdens, K.
- Text string matching to measure occupations in web-surveys; 2011; Tijdens, K.
- Web-based rating scales: HTML 5 and other innovations; 2011; Funke, F.
- A Methodological Inference towards the Quantification of Technological Frames ; 2011; Cachia, E., Camilleri, P.
- International Economics, Labour markets and Web Surveys Qualities; 2011; Guzi, M., de Pedraza, P.
- On Affordances and Technological Intersubjectivity; 2011; Vatrapu, R.
- The optimal number of response options in internet panel surveys; 2011; Thorsdottir, F.
- Online Research @ GESIS; 2011; Kaczmirek, L., Lenzner, T.
- Building online communities; 2011; Mlačić, B., Milas, G., Mikloušić, I.
- Romanian financing opportunities for the action; 2011; Moga, L.
- Current Projects at University of Ljubljana; 2011; Lozar Manfreda, K.
- 2nd WEBDATANET Meeting Amsterdam. Global Data Collecting; 2011; van Beveren, I.
- German Web-based Registry for Eating Disorders; 2011; Gross, G., Birgegård, A., Zipfel, S.
- Collecting information with Knowledge technologies; 2011; Foulonneau, M.
- E-dater, Artificial Actors, and German Households; 2011; Hebing, M.
- DIME-SHS; 2011; Lesnard, L.
- Web based Data Collection – A Jungle becoming a Field or a Field becoming a Jungle? ; 2011; Ole Finnemann, N.
- Online Surveys: Past, Present, and Future; 2011; de Leeuw, E. D.
- Vademecum: COST instruments, Core Group Meeting; 2011
- Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys 2011; 2011
- Calibrating Non-Probability Internet Samples with Probability Samples Using Early Adopter Characteristics...; 2011; DiSogra, C., Cobb, C. L., Chan, E., Dennis, J. M.
- A randomised trial and economic evaluation of the effect of response mode on response rate, response...; 2011; Scott, A., Jeon, S.-H., Joyce, C. M., Humphreys, J. S., Kalb, G., Witt, J., Leahy, A.
- Web Survey Methodology: Interface Design, Sampling and Statistical Inference; 2011; Couper, M. P.
- Globalpark Annual Market Research Software Survey 2010; 2011; Macer, T.; Wilson, S.
- Effect of interview modes on measurement of identity; 2011; Nandi, A., Platt, L.
- Maintaining Cross-Sectional Representativeness in a Longitudinal General Population Survey ; 2011; Lynn, P.
- Understanding Society Innovation Panel Wave 3: Results from Methodological Experiments; 2011; Burton, J., Budd, S., Gilbert, E., Jaeckle, A., McFall, S., Uhrig, S.C. N.
- The Effect of a Mixed Mode Wave on Subsequent Attrition in a Panel Survey: Evidence from the Understanding...; 2011; Lynn, P.
- Seeing Through the Eyes of the Respondent: An Eye-tracking Study on Survey Question Comprehension; 2011; Lenzner, A., Kaczmirek, L., Galesic, M.
- Eye Tracking in testing questionnaires: What’s the added value?; 2011; Tries, S.
- Panel Recruitment via Facebook; 2011; Toepoel, V.
- Dropout rates during completion of an occupation search tree in web-surveys; 2011; Tijdens, K.
- Usability and burden measurement in online forms; 2011; Thomsen, P.
- Dynamic Data Editing in online data collection for the Vacant Positions Survey; 2011; Stax, H.-P.