Web Survey Bibliography
Relevance & Research Question: A matrix question format (grid) has may advantages in traditional paper-and-pen surveys design: more questions can be presented in a smaller space, thereby saving paper, printing and mailing costs. With the onset of computer-assisted questionnaires and Internet research, these cost saving arguments are no longer valid. Still, matrix questions remain a widely used tool also in Web surveys (e.g. Couper, 2008). There is some empirical evidence (e.g. Toepoel et al 2005; Petchev, 2006; Callegaro, 2010) that matrix questions lead to more statisficing and more dropout. However, presenting questions as single items on the screen leads to longer response times and may increase response burden. In this study we therefore evaluate an alternative to traditional matrix and scrolling formats for web surveys: the horizontal scrolling matrix (HSM).
Methods & Data: In the HSM questions are presented one-by-one on the screen, while ease of answering is improved (e.g., after an answer is selected, the next question is quickly and automatically presented, there is no need for the next or previous button, etc.), and respondents still have an overview of the number of questions in the matrix through a visual bar. In a randomized field experiment we tested whether the HSM question format does improve data quality over the traditional matrix format. We used a 2 by 3 factorial design: the experimental factors were (1) question format: traditional matrix vs HSM and (2) length of ‘matrix’: 5, 10, and 15 items per matrix.
Results: The results were positive: the HSM-format led to fewer break-offs, better data quality and less satisficing as indicated by well-known response styles, such as non-differentiation and extremeness. The HSM helped improving data quality in the longer matrix conditions, which typically suffer from worse data quality. Finally, respondents evaluated the HSM format very positive.
Added Value: Online researchers can profit from the flexibility of web surveys by moving away from grid formats, which are a heritage of paper and pencil surveys. The HSM is a useful alternative, which prevents respondent demotivation by offering superior usability.
GOR Homepage (abstract) / (full text)
Web Survey Bibliography - The Netherlands (249)
- Adaptive survey designs to minimize survey mode effects. A case study on the Dutch Labour Force Survey...; 2013; Calinescu, M., Schouten, B.
- Measurement error calibration in mixed-mode sample surveys; 2013; Buelens, B., van der Brakel, J.
- Optimal Resource Allocation in Adaptive Survey Designs; 2013; Calinescu, M.
- Internet as a new source of information for the production of official statistics. Experiences of Statistics...; 2013; Heerschap, N.
- Web Panels for Official Statistics? ; 2013; Bethlehem, J., Cobben, F.
- Interactive applets on the Web for methods and statistics; 2013; Reips, U. -D., McClelland, G.
- Economic valuation in Web surveys; A review of the state of the art and best practices; 2013; Menegaki, A. N., Tsagarakis, K. P.
- Can creative web survey questionnaire design improve the response quality?; 2013; Angelovska, J., Mavrikiou, P. M.
- Methodological Issues in the Design of Online Surveys for Measuring Unethical Work Behavior: Recommendations...; 2013; Wouters, K., Maesschalck, J., Peeters, C. F. W., Roosen, M.
- Panel Conditioning in Difficult Attitudinal Questions; 2013; Binswanger, J., Schunk, D., Toepoel, V.
- Random versus Systematic Error in a Mixed Mode Online-Telephone Survey; 2013; Hox, J., Scherpenzeel, A., Boeve, A., Boeve, A., de Leeuw, E. D.
- Mobility and Smartphones: a pilot study of travel data collection among experienced and inexperienced...; 2013; Douhou, S., Scherpenzeel, A.
- Does It Pay Off to Include Non-Internet Households in an Internet Panel? ; 2013; Leenheer, J., Scherpenzeel, A.
- European Values Study - methodological and substantive applications; 2013; Luijkx, R., Jagodzinski, W.
- Mode Effects in Mixed-Mode Surveys: Prevention, Diagnostics, and Adjustment 1; 2013; de Leeuw, E. D., Dillman, D. A., Schouten, B.
- Scientific impact of the MESS Project: A brief overview; 2013; Das, M.
- Estimating Measurement Effects of Survey Modes From Between and Within Subject Designs; 2013; Klausch, L. T., Hox, J., Schouten, B.
- Comparing Survey Results Obtained via Mobile Devices and Computers: An Experiment With a Mobile Web...; 2013; de Bruijne, M., Wijnant, A.
- Measuring Wages Worldwide: Exploring the Potentials and Constraints of Volunteer Web Surveys; 2013; Steinmetz, S., Raess, D., Tijdens, K., de Pedraza, P.
- A standard for test reliability in group research; 2013; Ellis, J. L.
- The comparison of road safety survey answers between web-panel and face-to-face; Dutch results of SARTRE...; 2013; Goldenbeld, C., de Craen, S.
- Mixed-mode including web: Recent developments at Statistics Netherlands; 2013; Luiten, A., Schouten, B.
- The smartphone in survey research: experiments for time use data; 2013; Fernee, H., Scherpenzeel, A.
- Exploring Response Differences between Face-to-Face and Web Surveys: A Qualitative Comparative Analysis...; 2013; Bennink, M., Moors, G., Gelissen, J.
- Testing the Validity of Gender Ideology Items by Implementing Probing Questions ; 2013; Behr, D., Braun, M., Kaczmirek, L., Bandilla, W.
- Disentangling Mode-Specific Selection and Measurement Bias in Social Surveys; 2012; Buelens, B., van der Laan, J., Schouten, B., Klausch, L. T., van der Brakel, J., Burger, J.
- Sensitive topics in PC Web and mobile web surveys: Is there a difference?; 2012; Mavletova, A. M., Couper, M. P.
- Global market research 2012; 2012
- Online Surveys 2.0; 2012; Elferink, R.
- Especially for You: Motivating Respondents in an Internet Panel by Offering Tailored Questions; 2012; Oudejans, M.
- Improving the quality of complex surveys: The case of the EU Labour Force Survey ; 2012; van der Valk, J.
- 28 Questions to Help Buyers of Online Samples; 2012
- A Choice in Mode: A Solution for Increasing Response Rates of Hard-to-Survey Populations?; 2012; Haan, M., Ongena, Y. P.
- Better Answers to Basic Questions: Enhancing the accuracy of online reach and audience metrics; 2012; van Dam, P. H., van Ossenbruggen, R., Voorend, R.
- Probabilistic survey questions and incorrect answers: Retirement income replacement rates; 2012; van Santen, P., Alessie, R., Kalwij, A.
- Building Your Own Online Panel Via E-Mail and Other Digital Media; 2012; Toepoel, V.
- Disfluencies and Gaze Aversion in Unreliable Responses to Survey Questions; 2012; Schober, M. F., Conrad, F. G., Dijkstra, W., Ongena, Y. P.
- Recruiting A Probability Sample For An Online Panel: Effects Of Contact Mode, Incentives, And Information...; 2012; Scherpenzeel, A., Toepoel, V.
- Do Questions about Watching Internet Pornography Make People Watch Internet Pornography? A Comparison...; 2012; Peter, J., Valkenburg, P. M.
- “I think I know what you did last summer” Improving data quality in panel surveys; 2012; Lugtig, P. J.
- Tailoring the design of e-questionnaires to the response process: About audit trails and other methods...; 2012; Morren, M., Snijkers, G.
- Counting and Measuring Online: The Quality of Internet Surveys; 2012; de Leeuw, E. D.
- Time use data collection using Smartphones: Results of a pilot study among experienced and inexperienced...; 2012; Scherpenzeel, A., Sonck, N., Fernee, H.
- “What a waste of time!” vs “Why not participate?” On sentiments by business...; 2012; Torres van Grinsven, V., Snijkers, G., Daas, P.
- Challenges and pitfalls of measuring wages via web surveys - some explorations; 2012; Steinmetz, S., Bianchi, A., Tijdens, K., Biffignandi, S.
- Measuring work activities and skill requirements of occupations: Experiences from a European pilot study...; 2012; Tijdens, K., De Ruijter, E., De Ruijter, J.
- Innovation in online data collection for scientific research: the Dutch MESS project; 2012; Das, M.
- Clinicians’ Perspectives on a Web-Based System for Routine Outcome Monitoring in Old-Age Psychiatry...; 2012; Veerbeek, M. A., Voshaar, R. C. O., Pot, A. M.
- Question or Mode Effects in Mixed-Mode surveys: A Cross-cultural study in the Netherlands, Germany,...; 2012; de Leeuw, E. D., Nicolaas, G., Campanelli, P., Hox, J.
- Web Surveys: Methodological Problems and Research Perspectives; 2012; Biffignandi, S., Bethlehem, J.