Notice: the WebSM website has not been updated since the beginning of 2018.

Web Survey Bibliography

Title Response options order effect and category number association: An experiment using items on a five point satisfaction scale in a KnowledgePanel survey
Year 2008
Access date 15.09.2009

Previous research shows that the small changes in visual layout of questions can affect respondents’ answers. In this research, conducted with members of KnowledgePanel®, we manipulated the order of presentation of a fully labeled five point satisfaction scale (very satisfied to very unsatisfied); the association of numbers from 1 to 5 to each scale point; and we also reversed the number order (from 5 to 1). This manipulation created six experimental conditions to which 5,173 panel members were randomly allocated. For each condition, we presented seven different items on separate screens. The completion rate for this study, done in April 2008, was 71%. We investigated the differences between each version, especially focusing on the following comparisons: response options with numbers associated vs. options without numbers; order of the response option presentation; and order of the number association. Results from the experiment suggest that (1) reversing the order of presentation makes a significant difference resulting in primacy effects and in an increase of time latency when the scale is shown starting from “Very unsatisfied”. (2) Changing the number-label association does not make much of a difference; it appears that for a fully labeled scale the label is almost overriding the number. (3) Respondents are, however, slightly confused when the number-label association does not go in the expected direction. The results are discussed in light of current theories of question answering processes and visual layout.

Access/Direct link

Homepage (abstract)

Year of publication2008
Bibliographic typeConferences, workshops, tutorials, presentations
Full text availabilityAvailable on request

Web survey bibliography - Callegaro, M. (41)