Web Survey Bibliography
Effects of number of response options in web surveys: The role of verbal labels
Relevance and research question: Fully labelled agree/disagree rating scales are often used to obtain data pertaining to subjective phenomena in web surveys. Although the link between the number of response options and the quality of information obtained with rating scales is well established, the role of the verbal labels attached to the options is less well studied. The verbal lables define the length and the fineness of a fully labelled rating scale and adding response options may therefore either lengthen the scale and/or make it finer. The aim of the present work it to examine the impact of altering the number and verbal labels of response options on the quality of the information obtained. Fully labelled rating scales with five, seven and nine response options are examined. Verbal labels are either used to lengthen the scales or to make them finer.
Method and data: University students in Germany and Iceland were presented with six questions measuring attitude towards university education in a web survey. Respondents were randomly assigned to six experimental conditions. 1) Five response options with the labels strongly, somewhat and neither/nor, 2) five response options with the lables fully, somewhat and neither nor, 3) seven options with the labels fully, strongly, somewhat and neither nor, 4) seven response options with the labels strongly, somewhat, slightly and neither/nor, 5) seven response options with the labels fully, somewhat, slightly and neither/nor and finally 6) nine response options with the labels fully, strongly, somewhat, slightly and neither/nor.
Results: The results show that the effect of the number of response options on the quality of the information obtained depends on whehter the added response options lengthen the scale or make it finer. A finer scale seems to reduce non-differentiation and extreme response style.
Added value: The findings show that the verbal labels attached to the response options must be taken into account in empirical research on the optimal number of response options.
GOR Homepage (abstract) / (presentation)
Web Survey Bibliography - Usability, HCI (409)
- Tips for Evaluating Online Effectiveness; 2013; Stevenson, S. C.
- Using Web Surveys for Psychology Experiments: A Case Study in New Media Technology for Research; 2013; Peden, B. F., Tiry , A. M.
- The Distinctiveness of Online Research: Descriptive Assemblages, Unobtrusiveness, and Novel Kinds of...; 2013; Lanfrey, D.
- Advancing Research Methods with New Technologies; 2013; Sappleton, N.
- Compared to a small, supervised lab experiment, a large, unsupervised web-based experiment on a previously...; 2013; Ryan, R. S., Wilde, M., Crist, S.
- From mixed-mode to multiple devices. Web surveys, smartphone surveys and apps: has the respondent gone...; 2013; Callegaro, M.
- Moving an established survey online – or not?; 2013; Barber, T., Chilvers, D., Kaul, S.
- Using mobile devices to access the realities of youth: How identification with society influences political...; 2013; Smith, M.
- By the Numbers: Theory of adaptation or survival of the fittest?; 2013; Cavallaro, K.
- Modular Survey Design: A Bite Size Proposal; 2013; Kelly, F., Stevens, S., Johnson, A.
- Cyborgs vs. Monsters: Assembling Modular Surveys to Create Complete Datasets; 2013; Johnson, E. P., Siluk, L., Tarraf, S.
- Do I Have Your Full Attention?; 2013; Cape, P. J.
- Optimizing Surveys for Smartphones: Maximizing Response Rates While Minimizing Bias; 2013; Lattery, K., Park Bartolone, G., Saunders, T.
- Shorter Isn't Always Better; 2013; Burdein, I.
- Solving the Unintentional Mobile Challenge; 2013; Peterson, G., Mechling, J., LaFrance, J., Ham, G.
- Mobile Research Risk: What Happens to Data Quality When Respondents Use a Mobile Device for a Survey...; 2013; Baker-Prewitt, J.
- A standard for test reliability in group research; 2013; Ellis, J. L.
- The comparison of road safety survey answers between web-panel and face-to-face; Dutch results of SARTRE...; 2013; Goldenbeld, C., de Craen, S.
- Addressing Disclosure Concerns and Analysis Demands in a Real-Time Online Analytic System; 2013; Krenzke, T., Gentleman, J. F., Li, J., Moriarity, C.
- Examination of the equivalence of self-report survey-based paper-and-pencil and internet data collection...; 2013; Weigold, A., Weigold, I. K., Russell, E. J.
- Using Online and Paper Surveys - The Effectiveness of Mixed-Mode Methodology for Populations Over 50; 2013; De Bernardo, D. H., Curtis, A.
- Who responds to website visitor satisfaction surveys?; 2013; Andreadis, I.
- Comparison of psychometric properties of internet versions of the Marlowe-Crowne Social Desirability...; 2013; Vesteinsdottir, V., Reips, U. -D., Joinson, A. N., Porsdottir, F.
- Seducing the respondent – how to optimise invitations in on-site online research?; 2013; Póltorak, M., Kowalski, J.
- Influence of mobile devices in online surveys; 2013; Maxl, E., Baumgartner, T.
- The ONS Beyond 2011 Programme & possible implications for social surveys; 2013; Morris, L.
- Survey Research; 2013; Abbott, M. L., McKinney, J.
- The Use of E-Questionnaires in Organizational Surveys; 2013; Brender-Ilan, Y., Vinitzky, G.
- Online Survey Software; 2013; Baker, J. D.
- The effect of short formative diagnostic web quizzes with minimal feedback; 2013; Baelter, O., Enstroem, E., Klingenberg, B.
- Up Means Good: The Impact of Screen Position on Evaluative Ratings in Web Surveys.; 2013; Tourangeau, R., Conrad, F. G., Couper, M. P.
- What we can learn from unintentional mobile respondents; 2012; Peterson, G.
- The integration of facebook into class management: an exploratory study; 2012; Chou, P. N.
- The cross platform report. Q2 -2012 - US; 2012
- Mobile usability; 2012; Nielsen, J., Budiu, R.
- Smartphone Apps and User Engagement: Collecting Data in the Digital Era; 2012; Link, M. W.
- How Often Do You Use the App with a Bird on It? Exploring Differences in Survey Completion Times, Primacy...; 2012; Buskirk, T. D.
- Data quality of questions sensitive to social-desirability bias in web surveys; 2012; Lozar Manfreda, K., Zajc, N., Berzelak, N., Vehovar, V.
- Online Questionnaires: Development of ‘basic requirements’; 2012; Tries, S., Blanke, K.
- Social research in online context: methodological reflections on web surveys from a case study; 2012; Pandolfini, V.
- Improving Survey Website Usability ; 2012; Vannette, D.
- How accurate are surveys of objective phenomena?; 2012; Chang, L. C., Krosnick, J. A.
- Pros and cons of Internet based User Satisfaction Surveys; 2012; Consoli, A., Matsulevits, L.
- The re-engineering of the Structural Earnings survey process: Mixed - Mode data collection and new E...; 2012; Cardinaleschi, S., De Santis, S., Rocci, F., Spinelli, V.
- Between demand and reality: Ensuring efficiency and quality in pretesting questionnaires; 2012; Sattelberger, S., Blanke, K.
- How to provide high data quality in online-questionnaires: Setting guidelines in design; 2012; Tries, S., Nebel, S., Blanke, K.
- The Feasibility of Conducting a Web Survey Using Respondent Driven Sampling among Transgenders in the...; 2012; Kappelhof, J.
- Device Diversity: Understanding the complexity of varied devices for taking surveys – Case study...; 2012; Pearson, C., Backlund, K., Veling, L., Tsvelik, M., Jehoel, S.
- Research in the Mobile Mindset: Exploring the unexplored in the mobile research space; 2012; Willems, A., Veris, E., Verhaeghe, A.
- WebSM Study: Survey software features overview ; 2012; Vehovar, V.; Cehovin, G.; Kavcic, L.; Lenar, J.