Web Survey Bibliography
Relevance & Research Question: Optimizing online questionnaires for certain browsers not only raises the risk of loosing respondents but also may bias the sample composition. In their GOR paper on the low-tech principle, Buchanan and Reips (2001) found that more educated users were more likely to turn JavaScript off. Furthermore, respondents using Mac OS scored higher on the personality trait Openness to Experience than users of Windows OS. This study focuses on the question if these differences still hold, ten years later and with a sample of experienced Internet users.
Methods & Data: The questionnaire was a Big Five personality inventory. Following the low-tech principle, participation was possible with any Web browser.
Results: Overall, 2.6% of all participants (N = 358) had JavaScript disabled. Male respondents were more likely to have JavaScript disabled than female respondents, chi^2(1, N = 344) = 11.64, p = .001, odds ratio = 15.6. Mac users scored higher on Openness (e.g., “I enjoy hearing new ideas”) than Windows users, F(1, 294) = 12.14, p = .001, eta^2 = .040. Furthermore, respondents using Macs scored lower on Agreeableness (e.g., “I am interested in other people”) than users of Windows, F(1, 294) = 9,02, p = .003, eta^2 = .030. Within respondents running Windows Extraversion (e.g., “I am the life of the party”) was slightly higher for users of the Internet Explorer in comparison to users of Firefox, F(1, 277) = 4.13, p = .043, eta^2 = .015. Finally, there was a tendency that respondents with JavaScript deactivated scored lower on Openness than respondents with this technology activated, F(1, 312) = 3.05, p = .082, eta^2 = .010.
Added Value: The present study confirms and extends the results from Buchanan and Reips (2001). A questionnaire exclusively optimized for certain OSs or browsers can seriously bias the psychological and demographical sample composition. In the present study restricting participation to respondents with JavaScript enabled would have reduced the number of male participants. Overall, it is recommended either to refrain from using complex technologies or to implement alternative low-tech versions of questionnaires as fallback.
GOR Homepage (abstract)
Web survey bibliography - General Online Research Conference (GOR) 2012 (26)
- Is „chapterisation“ a viable alternative to traditional progress indicators ?; 2012; Spicer, R., Dowling, Z.
- Exploring New Pathways to Survey Recruitment; 2012; Bilgram, V., Stadler, D.Jawecki, G.
- Understanding selection bias in a worldwide, volunteer web-survey; 2012; Tijdens, K., Steinmetz, S.
- Does Mode Matter? Initial Evidence from the German Longitudinal Election Study (GLES); 2012; Blumenstiel, J. E., Rossmann, J.
- The Representativity of Web Surveys of the General Population compared to Traditional Modes and Mixed...; 2012; Klausch, L. T., Schouten, B., Hox, J.
- Surveytainment 2.0: Why investing 10 more minutes more in constructing your questionnaire is worth considering...; 2012; Muehle, A., Tress, F., Schmidt, S., Winkler, T.
- Market research online community (MROC) versus focus group; 2012; Zuber, M.
- Data quality in MAWI and CAWI; 2012; Mavletova, A. M., Blasius, J.
- Time use data collection using Smartphones: Results of a pilot study among experienced and inexperienced...; 2012; Scherpenzeel, A., Sonck, N., Fernee, H., Morren, Me.
- Scrutinizing Dynamics – Rolling panel waves in theory and practice; 2012; Faas, T., Blumenberg, J. N.
- Little experience with technology as a cause of nonresponse in online surveys; 2012; Struminskaya, B., Schaurer, I., Kaczmirek, L., Bandilla, W.
- Automatic Forwarding on Web Surveys – Some Outlines and Remarks; 2012; Selkaelae, A.
- Thinking, Planning & Operationalizing Empirical Mixed Methods Research Design; 2012; Ruhi, U.
- Continuous large-scale volunteer web-surveys: The experience of Lohnspiegel and WageIndicator; 2012; Oez, F.
- Is Pretesting Established Among Online Survey Tool Users?; 2012
- An Evaluation of Two Non-Reactive Web Questionnaire Pretesting Methods; 2012; Lenzner, T.
- Recommendations for implementing online surveys and simple experiments in social and behavioural research...; 2012; Hewson, C. M.
- High potential for mobile Web surveys: Findings from a survey representative for German Internet users...; 2012; Funke, F., Wachenfeld, A.
- A taxonomy of paradata for web surveys and computer assisted self interviewing (Casi); 2012; Callegaro, M.
- Can Social Media Research replace traditional research methods?; 2012; Faber, T., Einhorn, M., Hofmann, O., Loeffler, M.
- Bad Boy Matrix Question – Whatcha gonna do when they come for you?; 2012; Tress, F.
- Matrix vs. Single Question Formats in Web Surveys: Results from a large scale experiment; 2012; Klausch, L. T., de Leeuw, E. D., Hox, J., de Jongh, A., Roberts , A.
- Effects of Static versus Dynamic Formatting Instructions for Open-Ended Numerical Questions in Web Surveys...; 2012; Kunz, T., Fuchs, M.
- FamilyVote – Conducting online surveys with children and families; 2012; Geissler, H., Peeters, H.
- The influence of social desirability on data quality in face-to-face and web surveys; 2012; Keusch, F.
- Reducing the Threat of Sensitive Questions in Online Surveys; 2012; Couper, M. P.