Changing the alignment or type of rating scales is a simple way of expanding the possibilities for designing questionnaires. Flexibility in design is appreciated in Web questionnaires, for example to be able to optimize questionnaires according to the respondent's screen format (especially in mobile Web surveys to avoid bias that occurs when respondents have to scroll to see all response options). But there is a danger that changes in response scales can impair data quality.

We manipulated the available rating scale in a 2 (alignment) x 2 (type of scale) factorial between-subjects design. Alignment was either horizontal or vertical. For type of scale we used discrete ve-point categorical scales and continuous visual analogue scales (VASs). VASs were implemented as plain lines with 250 possible values and both ends verbally anchored. VASs are appreciated, e.g. in medical research, because small dierences can be detected within- or between-subjects. However, they are rarely used in the social and behavioral sciences, most probably because the analysis of data from non-computerized VASs is burdensome and because of a considerable lack of methodological research.

392 students participated in a Web experiment with a questionnaire consisting of 40 items on the Big Five dimensions of personality. Neither type of scale nor alignment mattered regarding dropout and lurking (i.e. participants rating not a single item at all). Missing data occurred more often with 5-point scales than with VASs, Chi2(2, N = 391) = 5.24, p < .05, alignment did not matter. Neither alignment nor scale type had a statistically signi cant eect on means of the ve dimensions of personality.

Thanks to computerized data collection we were able collect two paradata: response time and frequency of modi ed answers. Neither type of scale nor alignment showed an in uence on response time. With VASs ratings were modi ed more often than with 5-point scales, F(1, 390) = 49.83, p < .001, eta2 = .114. Alignment had no in uence on frequency of modi ed answers. There were no interactions.

Five items (one from every dimension of personality) were repeated at the end of the questionnaire. We found test-retest reliability (CronbachOs alpha) to be a little higher with 5-point scales (M = .92) than with VASs (M = .90). Horizontal alignment led to higher scores (M = .92) than vertical alignment (M = .90). Consequently, horizontal 5-point scales led to the highest reliability scores (M = .93) and vertical VASs performed worse (M = .88).

Overall, alignment can apparently be used as a means of design with ve-point categorical scales as well as with VASs without running the risk of seriously biasing data. However, it should be kept in mind that scales of short to medium length were used in this study. Further research with longer scales is needed.