Web Survey Bibliography
Effects of number of response options in web surveys: The role of verbal labels
Relevance and research question: Fully labelled agree/disagree rating scales are often used to obtain data pertaining to subjective phenomena in web surveys. Although the link between the number of response options and the quality of information obtained with rating scales is well established, the role of the verbal labels attached to the options is less well studied. The verbal lables define the length and the fineness of a fully labelled rating scale and adding response options may therefore either lengthen the scale and/or make it finer. The aim of the present work it to examine the impact of altering the number and verbal labels of response options on the quality of the information obtained. Fully labelled rating scales with five, seven and nine response options are examined. Verbal labels are either used to lengthen the scales or to make them finer.
Method and data: University students in Germany and Iceland were presented with six questions measuring attitude towards university education in a web survey. Respondents were randomly assigned to six experimental conditions. 1) Five response options with the labels strongly, somewhat and neither/nor, 2) five response options with the lables fully, somewhat and neither nor, 3) seven options with the labels fully, strongly, somewhat and neither nor, 4) seven response options with the labels strongly, somewhat, slightly and neither/nor, 5) seven response options with the labels fully, somewhat, slightly and neither/nor and finally 6) nine response options with the labels fully, strongly, somewhat, slightly and neither/nor.
Results: The results show that the effect of the number of response options on the quality of the information obtained depends on whehter the added response options lengthen the scale or make it finer. A finer scale seems to reduce non-differentiation and extreme response style.
Added value: The findings show that the verbal labels attached to the response options must be taken into account in empirical research on the optimal number of response options.
GOR Homepage (abstract) / (presentation)
Web Survey Bibliography - Education (147)
- On the Use of Latent Variable Models to Detect Differences in the Interpretation of Vague Quantifiers...; 2013; Griffin, J.
- Using interactive feedback to enhance response quality in Web surveys. The case of open-ended questions...; 2013; Emde, M., Fuchs, M.
- Reducing Response Order Effects in Check-All-That-Apply Questions by Use of Dynamic Tooltip Instructions...; 2013; Kunz, T., Fuchs, M.
- Ethics, privacy and data security in web-based course evaluation; 2013; Salaschek, M., Meese, C., Thielsch, M.
- Speed (necessarily) doesn’t kill: A new way to detect survey satisficing; 2012; Garland, P. et al.
- Social research in online context: methodological reflections on web surveys from a case study; 2012; Pandolfini, V.
- What can be said about quality in the Central Population Register based on a self-completion survey...; 2012; Falnes-Dalheim, E., Pedersen, H. E.
- Boosting Web pick-up Rates by referring to Compliance Principles ; 2012; Falnes-Dalheim, E., Haraldsen, G., Sundvoll, A.
- Ebook readings jumps, print book reading declines; 2012; Rainie, L., Duggan, M.
- Prenotification, Incentives, and Survey Modality: An Experimental Test of Methods to Increase Survey...; 2012; Tepper, J. R., Jacob, B.
- Does survey experience affect respondents’ reported level of satisfaction?; 2012; Schultz Christensen, A., Ladenburg, J.
- Nonresponse and Online Student Evaluations of Teaching: Understanding the Inﬂuence of Salience...; 2012; Adams, M. J. D., Umbach, P. D.
- What are the Odds? Lotteries versus Cash Incentives. Response Rates, Cost and Data Quality for a Web...; 2012; Stevenson, J., Dykema, J., Klein, L., Cyffka, K., Goldrick-Rab, S.
- Exploring the impact of animation-based questionnaire on conducting a web-based educational survey and...; 2012; Chien, Y.-T., Chang, C.-Y.
- Effects of number of response options in web surveys: The role of verbal labels; 2012; Thorsdottir, F., Fuchs, M., Jonsdottir, J.
- Asking Sensitive Questions in Online Surveys: An Experimental Comparison of the Randomized Response...; 2012; Diekmann, A., Hoeglinger, M., Jann, B.
- Use of incentives and web-based administration for surveying student alcohol and substance use in an...; 2012; Onoye, J. M., Goebert, D., Nishimura, S.
- Toward wiser public judgment; 2011; Yankelovich, D., Friedman, W.
- GRE® program announces big benefits and big savings for GRE® test takers worldwide; 2011
- A course is a course is a course: Factor invariance in student evaluation of online, blended and face...; 2011; Dziuban, C. D., Moskal, P.
- A National Profile of Teacher Education Faculty: The Construction of an Online Survey; 2011; Graber, K. C., Erwin, H., Woods, A. M., Rhoades, J., Zhu, W.
- How do Respondents Perceive a Questionnaire? The Contribution of Open-ended Questions; 2011; Markou, E., Garnier, B.
- In the Face of Declining Budgets: The Student Experience at Washington State University ; 2011; Allen, T., Dillman, D. A., Garza, B., Millar, M. M.
- Examining Response Rates and Patterns in a Multimode Experiment: A Study of Department Chairs/Heads...; 2011; Foster, K. N., Gaughan, M.
- Exploring Visual Design Effects In Web Survey Questions. An Experimental Study Using Animated Answer...; 2011; Emde, M., Fuchs, M.
- Display Matters: A Test of Visual Display Options in a Web-Based Survey.; 2011; Romano Bergstrom, J. C., Chen, J. M., Gilbert, T. R., Jans, M.
- Minimizing Item Nonresponse in a School-based Web Survey of High School Students.; 2011; Burns, L., Dever, J.
- Which Technologies Do Respondents Use in Online Surveys – An International Comparison?; 2011; Kaczmirek, L., Behr, D., Bandilla, W.
- Mode Effect Analysis - Paper Respondents vs. Web Respondents in the 2004-05 Teacher Follow-up Survey; 2011; Aritomi, T., Hill, J.
- An experiment to test the feasibility and quality of a web-based questionnaire of teachers; 2011; Tepper, J. R.
- Sponsor Prominence and Responses Patterns to an Online Survey; 2011; Boulianne, J. S., Klofstad, C. A., Basson, D.Boulianne, S. J.
- The weirdest people in the world?; 2010; Heine, S. J., Henrich, J., Norenzayan, A.
- Codebook and explanatory note on the WageIndicator dataset ; 2010; Tijdens, K., van Zijl, S., Hughie-Williams, M., van Klaveren, M., Steinmetz, S.
- Web Surveys when Databases have Incomplete Email Records: The Impact of Modifying the Method of Respondent...; 2010; Elasmar, M. G.
- An Experiment to Test the Feasibility and Quality of a Web-Based Questionnaire of Teachers; 2010; Jacob, R., Scott, L., Rowan, B.
- Mode Effect Analysis: Paper respondents vs. Web respondents in the 2004-05 Teacher Follow-up Survey; 2010; Aritomi, T., Hill, J.
- L’uso di Internet nella ricerca sociale: vantaggi e svantaggi di una web survey; 2010; Pandolfini, V.
- The impact of incentives and interview methods on response quantity and quality in diary- and booklet...; 2010; Bonke, J., Fallesen, P.
- Color red in web-based knowledge testing; 2010; Gnambs, T., , Batinic, B.Appel, M.
- Exploring the feasibility of an online contextualised animation-based questionnaire for educational...; 2010; Chien, Y.-T., Chang, C.-Y.
- Does Visual Appeal Matter? Effects of Web Survey Aesthetics on Survey Quality; 2010; Mahon-Haft, T., Dillman, D. A.
- Differences in responses to Web and paper surveys among school professionals; 2010; Yetter, G., Capaccioli, K.
- Web-based versus paper-based data collection for the evaluation of teaching activity: empirical evidence...; 2010; Lalla, M., Ferrari, D.
- Response format effects on measurement of employment; 2009; Thomas, R. K., Dillman, D. A., Smyth, J. D.
- College Experiences Survey: Methodological Summary. Final Report; 2009; DesRoches, D., Hall, J. W., Santos, B.
- Exploration of secondary students’ creativity by integrating web-based technology into an innovative...; 2009; Jang, S.- J.
- An experiment on the effects of non-response reweighting on estimators' precision in a web survey; 2009; Fabrizi, E., Biffignandi, S., Toninelli, D.
- A recipe for effective participation rates for web-based surveys ; 2009; Bennett, L.; Nair, C. S.
- Informal Learning in Virtual Communities. Individual Practice between Information Retrieval, Observation...; 2009; Kahnwald, N.
- Engaging faculty through collection development utilizing online survey tools ; 2009; Jensen, K.