Notice: the WebSM website has not been updated since the beginning of 2018.

Web Survey Bibliography

Title Focus on mobile surveys: Do the number of scale points and scale order affect rating scale results?
Year 2017
Access date 13.04.2017

1. Relevance & Research Question:

Scale types (straight or odd scale points) and scale order (i.e. the order in which response options of a rating scale are presented) were frequently investigated in the past, since the question arose whether the results of surveys are influenced by the nature and application of a scale. However, experimental studies have not yielded clear and robust, but rather contradictory and unambiguous results. Considering a strong trend towards online surveys in general and mobile research in particular, this question appears in a new light (reduced interview time and visibility).

2. Methods & Data:

In order to analyze the influence on rating scales both mentioned factors, scale type as well as scale order an experimental online survey design was created (n=600, Germany, 16+ years), covering a 5 point vs. 6 point satisfaction scale, each with two different orders (positive-negative vs. negative-positive scale; 2*2 design). Subjects were asked to evaluate 5 facets of the interview (topic of the interview etc.). Four test groups were randomly generated, each receiving one specific scale form.

3. Results:

Based on the experimental setup, no significant differences between a 6-point and a 5-point scale can be identified as long as the scale direction is positive to negative, i.e. the best rating ("very satisfied") is shown left next to the statement/item. Particularly strong influences on the results are apparent when using a in the 6-point scale and changing the scale direction to a negative to positive order. In this case the evaluation of the interview is significantly worse than in test groups with a positive-negative ordered scale. Furthermore, this effect is particularly strong in a subgroup of respondents, which used a smartphone to participate in the web survey.

4. Added Value:

Firstly, it is irritating that the mean values on a 6-scale scale differ by more than 0.5 points depending on scale order. Secondly, it is disturbing that the effects increase when respondent used a smartphone. Thirdly, this has strong implications for developing a suitable questionnaire design for online / mobile studies.

Year of publication2017
Bibliographic typeConferences, workshops, tutorials, presentations

Web survey bibliography (8390)