Relevance & Research Question: Due to the rising penetration rate of smart phones, web surveys are increasingly accessed and completed on smart phones or tablets instead of personal computers. In addition, mobile web surveys are gaining popularity as an alternative self-administered data collection mode. However, the limited amount of space for displaying questions and scales on mobile screens poses new challenges for survey researchers. One of them is the optimal design of response scales. Survey literature has shown that design features of response scales affect survey responses and that answers to rating scales differ between web and mobile web respondents. This paper focuses on one design feature of a response scale – the direction of scale – and studies the impact of the direction of response scale on the actual answers. Drawing on empirical findings from web studies, we hypothesize a stronger scale direction effect among mobile web respondents than PC web respondents.
Methods & Data: We are conducting a methodological experiment crossing three factors (mode of administration, scale direction, scale alignment) in a full factorial design. Half of the sampled are invited to complete a web survey on their personal computer and the other half are invited to fill out the same questionnaire on their smartphone. All rating scales are shown to run either from positive to negative (or high to low) or from negative to positive (or low to high). In addition, all response scales are presented either horizontally, vertically, or as a drop-down menu. We recruit respondents who have access to the internet both on a computer and through their own smart phones via MTurk and Craigslist.
Results: The experiment is currently in the field and first results will be available in January 2014.
Added Value: Our study is the first to experimentally manipulate both the direction and alignment of rating scales in a web and a mobile web survey. Given the increasing interest in self-administered data collection through mobile devices, the results of the study will be of high relevance and significance to academic and non-academic survey researchers who consider collecting data through mobile devices.