Web Survey Bibliography
Relevance & Research Question: A matrix question format (grid) has may advantages in traditional paper-and-pen surveys design: more questions can be presented in a smaller space, thereby saving paper, printing and mailing costs. With the onset of computer-assisted questionnaires and Internet research, these cost saving arguments are no longer valid. Still, matrix questions remain a widely used tool also in Web surveys (e.g. Couper, 2008). There is some empirical evidence (e.g. Toepoel et al 2005; Petchev, 2006; Callegaro, 2010) that matrix questions lead to more statisficing and more dropout. However, presenting questions as single items on the screen leads to longer response times and may increase response burden. In this study we therefore evaluate an alternative to traditional matrix and scrolling formats for web surveys: the horizontal scrolling matrix (HSM).
Methods & Data: In the HSM questions are presented one-by-one on the screen, while ease of answering is improved (e.g., after an answer is selected, the next question is quickly and automatically presented, there is no need for the next or previous button, etc.), and respondents still have an overview of the number of questions in the matrix through a visual bar. In a randomized field experiment we tested whether the HSM question format does improve data quality over the traditional matrix format. We used a 2 by 3 factorial design: the experimental factors were (1) question format: traditional matrix vs HSM and (2) length of ‘matrix’: 5, 10, and 15 items per matrix.
Results: The results were positive: the HSM-format led to fewer break-offs, better data quality and less satisficing as indicated by well-known response styles, such as non-differentiation and extremeness. The HSM helped improving data quality in the longer matrix conditions, which typically suffer from worse data quality. Finally, respondents evaluated the HSM format very positive.
Added Value: Online researchers can profit from the flexibility of web surveys by moving away from grid formats, which are a heritage of paper and pencil surveys. The HSM is a useful alternative, which prevents respondent demotivation by offering superior usability.
GOR Homepage (abstract) / (full text)
Web Survey Bibliography - The Netherlands (226)
- Comparing Survey Results Obtained via Mobile Devices and Computers: An Experiment With a Mobile Web...; 2013; de Bruijne, M., Wijnant, A.
- Measuring Wages Worldwide: Exploring the Potentials and Constraints of Volunteer Web Surveys; 2013; Steinmetz, S., Raess, D., Tijdens, K., de Pedraza, P.
- A standard for test reliability in group research; 2013; Ellis, J. L.
- The comparison of road safety survey answers between web-panel and face-to-face; Dutch results of SARTRE...; 2013; Goldenbeld, C., de Craen, S.
- Mixed-mode including web: Recent developments at Statistics Netherlands; 2013; Luiten, A., Schouten, B.
- The smartphone in survey research: experiments for time use data; 2013; Fernee, H., Scherpenzeel, A.
- Exploring Response Differences between Face-to-Face and Web Surveys: A Qualitative Comparative Analysis...; 2013; Bennink, M., Moors, G., Gelissen, J.
- Sensitive topics in PC Web and mobile web surveys: Is there a difference?; 2012; Mavletova, A. M., Couper, M. P.
- Global market research 2012; 2012
- Online Surveys 2.0; 2012; Elferink, R.
- Especially for You: Motivating Respondents in an Internet Panel by Offering Tailored Questions; 2012; Oudejans, M.
- Improving the quality of complex surveys: The case of the EU Labour Force Survey ; 2012; van der Valk, J.
- 28 Questions to Help Buyers of Online Samples; 2012
- A Choice in Mode: A Solution for Increasing Response Rates of Hard-to-Survey Populations?; 2012; Haan, M., Ongena, Y. P.
- Better Answers to Basic Questions: Enhancing the accuracy of online reach and audience metrics; 2012; van Dam, P. H., van Ossenbruggen, R., Voorend, R.
- Probabilistic survey questions and incorrect answers: Retirement income replacement rates; 2012; van Santen, P., Alessie, R., Kalwij, A.
- Building Your Own Online Panel Via E-Mail and Other Digital Media; 2012; Toepoel, V.
- Disfluencies and Gaze Aversion in Unreliable Responses to Survey Questions; 2012; Schober, M. F., Conrad, F. G., Dijkstra, W., Ongena, Y. P.
- Recruiting A Probability Sample For An Online Panel: Effects Of Contact Mode, Incentives, And Information...; 2012; Scherpenzeel, A., Toepoel, V.
- Do Questions about Watching Internet Pornography Make People Watch Internet Pornography? A Comparison...; 2012; Peter, J., Valkenburg, P. M.
- “I think I know what you did last summer” Improving data quality in panel surveys; 2012; Lugtig, P. J.
- Tailoring the design of e-questionnaires to the response process: About audit trails and other methods...; 2012; Morren, M., Snijkers, G.
- Counting and Measuring Online: The Quality of Internet Surveys; 2012; de Leeuw, E. D.
- Time use data collection using Smartphones: Results of a pilot study among experienced and inexperienced...; 2012; Scherpenzeel, A., Sonck, N., Fernee, H.
- “What a waste of time!” vs “Why not participate?” On sentiments by business...; 2012; Torres van Grinsven, V., Snijkers, G., Daas, P.
- Challenges and pitfalls of measuring wages via web surveys - some explorations; 2012; Steinmetz, S., Bianchi, A., Tijdens, K., Biffignandi, S.
- Measuring work activities and skill requirements of occupations: Experiences from a European pilot study...; 2012; Tijdens, K., De Ruijter, E., De Ruijter, J.
- Innovation in online data collection for scientific research: the Dutch MESS project; 2012; Das, M.
- Clinicians’ Perspectives on a Web-Based System for Routine Outcome Monitoring in Old-Age Psychiatry...; 2012; Veerbeek, M. A., Voshaar, R. C. O., Pot, A. M.
- Question or Mode Effects in Mixed-Mode surveys: A Cross-cultural study in the Netherlands, Germany,...; 2012; de Leeuw, E. D., Nicolaas, G., Campanelli, P., Hox, J.
- Web Surveys: Methodological Problems and Research Perspectives; 2012; Biffignandi, S., Bethlehem, J.
- Understanding selection bias in a worldwide, volunteer web-survey; 2012; Tijdens, K., Steinmetz, S.
- The Representativity of Web Surveys of the General Population compared to Traditional Modes and Mixed...; 2012; Klausch, L. T., Schouten, B., Hox, J.
- Time use data collection using Smartphones: Results of a pilot study among experienced and inexperienced...; 2012; Scherpenzeel, A., Sonck, N., Fernee, H., Morren, M.
- Matrix vs. Single Question Formats in Web Surveys: Results from a large scale experiment; 2012; Klausch, L. T., de Leeuw, E. D., Hox, J., de Jongh, A., Roberts , A.
- Reminders in Web-Based Data Collection: Increasing Response at the Price of Retention?; 2012; Goeritz, A., Crutzen, R.
- Response rate of bricklayers and supervisors on an internet or a paper-and-pencil questionnaire; 2012; Boschman, J. S., Frings-Dresen, M. H. W., Sluiter, J. K., van der Molen, H. F.
- Testing the Validity of Gender Ideology Items by Implementing Probing Questions ; 2012; Behr, D., Braun, M., Kaczmirek, L., Bandilla, W.
- Global market research 2011; 2011
- Essential methods for design based sample surveys; 2011; Pfeffermann, D., Rao, C. R.
- Text string matching to measure occupations in web-surveys; 2011; Tijdens, K.
- 2nd WEBDATANET Meeting Amsterdam. Global Data Collecting; 2011; van Beveren, I.
- Web based Data Collection – A Jungle becoming a Field or a Field becoming a Jungle? ; 2011; Ole Finnemann, N.
- Online Surveys: Past, Present, and Future; 2011; de Leeuw, E. D.
- Panel Recruitment via Facebook; 2011; Toepoel, V.
- Dropout rates during completion of an occupation search tree in web-surveys; 2011; Tijdens, K.
- Disentangling relative mode effects for the web survey mode in the Safety Monitor; 2011; Schouten, B., van de Brakel, J., Buelens, B., Klausch, L. T., van der Laan, J.
- Combining web surveys and XBRL ; 2011; Roos, M.
- Developing Electronic Questionnaires at Statistics Canada: Experiences and Challenges in a Changing...; 2011; Lawrence, D.
- Effects of internet data collection in business surveys – the case of the Dutch SBS; 2011; Giesen, D.