Notice: the WebSM website has not been updated since the beginning of 2018.

Web Survey Bibliography

Title The effect of varying the number of response alternatives in rating scales: Experimental evidence from intra-individual effects
Source Behavior Research Methods, 41, 2, pp. 295-308
Year 2009
Access date 07.07.2009

Despite a hundred years of questionnaire testing, no consensus has been reached on the optimal number of response alternatives in rating scales. Differences in prior research may have been due to the use of various psychometric models (classical test theory, item factor analysis, and item response theory) and different performance criteria (reliability, convergent/discriminant validity, and internal structure of the questionnaire). Furthermore, previous empirical studies on this issue have tackled the experimental design from a between-subjects perspective, thus ignoring intra-individual effects. In contrast with this approach, we propose a within-subjects experimental design and a comprehensive statistical methodology using structural equation models for studying all of these aspects simultaneously, therefore increasing statistical power. To illustrate the method, two personality questionnaires were examined using a repeated measures design. Results indicated that as the number of response alternatives increased, (1) internal consistency increased, (2) there was no effect on convergent validity, and (3) goodness of fit worsened. Finally, the article assesses the practical consequences of this research for the design of future personality questionnaires.

Access/Direct link

Journal Homepage (abstract)

Year of publication2009
Bibliographic typeJournal article
Full text availabilityAvailable on request

Web survey bibliography - Behavior Research Methods (16)