Notice: the WebSM website has not been updated since the beginning of 2018.

Web Survey Bibliography

Title The effect of horizontal and vertical scales on the response behavior when switching to a mobile first design
Year 2017
Access date 06.04.2017

Relevance & Research Question: The aim of this paper is to analyze the effects of horizontal and vertical scales on response behavior for smartphone versus tablet/desktop participants in the German Internet Panel (GIP). Changes in the way people use technology, in particular smartphones, affect the measurement quality of online surveys, which increasingly become mixed-device surveys. On smartphone screens, for example, it is difficult to display horizontal scales, forcing survey designers to rethink the way answer options are displayed. Moving from traditional horizontal scales to smartphone-compatible vertical scales may, however, impact on time-series of established measures, if respondents on desktops and/or smartphones answer differently on a horizontal than on a vertical scale.

Research questions of interest are for example:

Does the scale alignment have an effect on response behavior? In particular, do we find differential distribution effects across different alignments? And to what extent does the scale alignment affect item nonresponse and response times? Most importantly, are these effects different on desktops/tablets and smartphones?

Data: 59 experiments across six waves of the Germany Internet Panel, almost 3198 respondents per wave.

Methods: ANOVAs, (multilevel) (logistic) regressions.

Results: First analyses show no effect of scale alignment on variable distributions and response times for smartphone and desktop/tablet participants indicating an unproblematic switch from horizontal to vertical scales. However, analyses into distribution effects delivered significant results choosing extreme points when presented with a vertical scale as opposed to a horizontal scale.

Added Value: The participation in online surveys via smartphones is increasing but horizontal scales are still a common practice when designing questionnaires. Thus, it is of great importance to identify the effects of scale alignment on response behavior. Our experiments ensure a comprehensive investigation of the effect of scale alignment in 59 experiments implemented across six waves of the GIP and almost 3198 respondents per wave.

Year of publication2017
Bibliographic typeConferences, workshops, tutorials, presentations