Web Survey Bibliography
Relevance & Research Question: A matrix question format (grid) has may advantages in traditional paper-and-pen surveys design: more questions can be presented in a smaller space, thereby saving paper, printing and mailing costs. With the onset of computer-assisted questionnaires and Internet research, these cost saving arguments are no longer valid. Still, matrix questions remain a widely used tool also in Web surveys (e.g. Couper, 2008). There is some empirical evidence (e.g. Toepoel et al 2005; Petchev, 2006; Callegaro, 2010) that matrix questions lead to more statisficing and more dropout. However, presenting questions as single items on the screen leads to longer response times and may increase response burden. In this study we therefore evaluate an alternative to traditional matrix and scrolling formats for web surveys: the horizontal scrolling matrix (HSM).
Methods & Data: In the HSM questions are presented one-by-one on the screen, while ease of answering is improved (e.g., after an answer is selected, the next question is quickly and automatically presented, there is no need for the next or previous button, etc.), and respondents still have an overview of the number of questions in the matrix through a visual bar. In a randomized field experiment we tested whether the HSM question format does improve data quality over the traditional matrix format. We used a 2 by 3 factorial design: the experimental factors were (1) question format: traditional matrix vs HSM and (2) length of ‘matrix’: 5, 10, and 15 items per matrix.
Results: The results were positive: the HSM-format led to fewer break-offs, better data quality and less satisficing as indicated by well-known response styles, such as non-differentiation and extremeness. The HSM helped improving data quality in the longer matrix conditions, which typically suffer from worse data quality. Finally, respondents evaluated the HSM format very positive.
Added Value: Online researchers can profit from the flexibility of web surveys by moving away from grid formats, which are a heritage of paper and pencil surveys. The HSM is a useful alternative, which prevents respondent demotivation by offering superior usability.
GOR Homepage (abstract) / (full text)
Web Survey Bibliography - Usability, HCI (409)
- Tips for Evaluating Online Effectiveness; 2013; Stevenson, S. C.
- Using Web Surveys for Psychology Experiments: A Case Study in New Media Technology for Research; 2013; Peden, B. F., Tiry , A. M.
- The Distinctiveness of Online Research: Descriptive Assemblages, Unobtrusiveness, and Novel Kinds of...; 2013; Lanfrey, D.
- Advancing Research Methods with New Technologies; 2013; Sappleton, N.
- Compared to a small, supervised lab experiment, a large, unsupervised web-based experiment on a previously...; 2013; Ryan, R. S., Wilde, M., Crist, S.
- From mixed-mode to multiple devices. Web surveys, smartphone surveys and apps: has the respondent gone...; 2013; Callegaro, M.
- Moving an established survey online – or not?; 2013; Barber, T., Chilvers, D., Kaul, S.
- Using mobile devices to access the realities of youth: How identification with society influences political...; 2013; Smith, M.
- By the Numbers: Theory of adaptation or survival of the fittest?; 2013; Cavallaro, K.
- Modular Survey Design: A Bite Size Proposal; 2013; Kelly, F., Stevens, S., Johnson, A.
- Cyborgs vs. Monsters: Assembling Modular Surveys to Create Complete Datasets; 2013; Johnson, E. P., Siluk, L., Tarraf, S.
- Do I Have Your Full Attention?; 2013; Cape, P. J.
- Optimizing Surveys for Smartphones: Maximizing Response Rates While Minimizing Bias; 2013; Lattery, K., Park Bartolone, G., Saunders, T.
- Shorter Isn't Always Better; 2013; Burdein, I.
- Solving the Unintentional Mobile Challenge; 2013; Peterson, G., Mechling, J., LaFrance, J., Ham, G.
- Mobile Research Risk: What Happens to Data Quality When Respondents Use a Mobile Device for a Survey...; 2013; Baker-Prewitt, J.
- A standard for test reliability in group research; 2013; Ellis, J. L.
- The comparison of road safety survey answers between web-panel and face-to-face; Dutch results of SARTRE...; 2013; Goldenbeld, C., de Craen, S.
- Addressing Disclosure Concerns and Analysis Demands in a Real-Time Online Analytic System; 2013; Krenzke, T., Gentleman, J. F., Li, J., Moriarity, C.
- Examination of the equivalence of self-report survey-based paper-and-pencil and internet data collection...; 2013; Weigold, A., Weigold, I. K., Russell, E. J.
- Using Online and Paper Surveys - The Effectiveness of Mixed-Mode Methodology for Populations Over 50; 2013; De Bernardo, D. H., Curtis, A.
- Who responds to website visitor satisfaction surveys?; 2013; Andreadis, I.
- Comparison of psychometric properties of internet versions of the Marlowe-Crowne Social Desirability...; 2013; Vesteinsdottir, V., Reips, U. -D., Joinson, A. N., Porsdottir, F.
- Seducing the respondent – how to optimise invitations in on-site online research?; 2013; Póltorak, M., Kowalski, J.
- Influence of mobile devices in online surveys; 2013; Maxl, E., Baumgartner, T.
- The ONS Beyond 2011 Programme & possible implications for social surveys; 2013; Morris, L.
- Survey Research; 2013; Abbott, M. L., McKinney, J.
- The Use of E-Questionnaires in Organizational Surveys; 2013; Brender-Ilan, Y., Vinitzky, G.
- Online Survey Software; 2013; Baker, J. D.
- The effect of short formative diagnostic web quizzes with minimal feedback; 2013; Baelter, O., Enstroem, E., Klingenberg, B.
- Up Means Good: The Impact of Screen Position on Evaluative Ratings in Web Surveys.; 2013; Tourangeau, R., Conrad, F. G., Couper, M. P.
- What we can learn from unintentional mobile respondents; 2012; Peterson, G.
- The integration of facebook into class management: an exploratory study; 2012; Chou, P. N.
- The cross platform report. Q2 -2012 - US; 2012
- Mobile usability; 2012; Nielsen, J., Budiu, R.
- Smartphone Apps and User Engagement: Collecting Data in the Digital Era; 2012; Link, M. W.
- How Often Do You Use the App with a Bird on It? Exploring Differences in Survey Completion Times, Primacy...; 2012; Buskirk, T. D.
- Data quality of questions sensitive to social-desirability bias in web surveys; 2012; Lozar Manfreda, K., Zajc, N., Berzelak, N., Vehovar, V.
- Online Questionnaires: Development of ‘basic requirements’; 2012; Tries, S., Blanke, K.
- Social research in online context: methodological reflections on web surveys from a case study; 2012; Pandolfini, V.
- Improving Survey Website Usability ; 2012; Vannette, D.
- How accurate are surveys of objective phenomena?; 2012; Chang, L. C., Krosnick, J. A.
- Pros and cons of Internet based User Satisfaction Surveys; 2012; Consoli, A., Matsulevits, L.
- The re-engineering of the Structural Earnings survey process: Mixed - Mode data collection and new E...; 2012; Cardinaleschi, S., De Santis, S., Rocci, F., Spinelli, V.
- Between demand and reality: Ensuring efficiency and quality in pretesting questionnaires; 2012; Sattelberger, S., Blanke, K.
- How to provide high data quality in online-questionnaires: Setting guidelines in design; 2012; Tries, S., Nebel, S., Blanke, K.
- The Feasibility of Conducting a Web Survey Using Respondent Driven Sampling among Transgenders in the...; 2012; Kappelhof, J.
- Device Diversity: Understanding the complexity of varied devices for taking surveys – Case study...; 2012; Pearson, C., Backlund, K., Veling, L., Tsvelik, M., Jehoel, S.
- Research in the Mobile Mindset: Exploring the unexplored in the mobile research space; 2012; Willems, A., Veris, E., Verhaeghe, A.
- WebSM Study: Survey software features overview ; 2012; Vehovar, V.; Cehovin, G.; Kavcic, L.; Lenar, J.