Two different formats of labelling in rating scales are commonly used in questionnaires: verbal labels for the end categories only (END form), and verbal labels for each of the categories (ALL form). On the one hand, there are a lot of research findings which recommend the usage of the ALL form, but on the other hand, there is no clear evidence whether this only holds true for short answer scales (e.g. with up to five categories) or also for longer answer scales (e.g. with seven categories).
In the current study, we take a closer look at how much attention respondents pay to the information transported by verbal labels. Attention was tracked in a laboratory setting employing Web-based surveys and eye-tracking technology, measuring fixation times and fixation counts on answer scales. Additional dependent variables were ease of administration and measurement quality. We conducted two experiments: one applied seven and the other applied five answer categories with N = 47 participants in each of the experiments. The results show that the ALL form provides a better reliability, however, the probability that respondents attend to all verbal labels is significantly lower in the ALL form than in the END form. This was particularly evident in the case of the seven categories rating scale. With regard to the ease of administration mixed results were obtained. In decisions on the optimal design of rating scales the positive or negative wording of the items and the length of the item battery seem to play a role.