Notice: the WebSM website has not been updated since the beginning of 2018.

Web Survey Bibliography

Title The Effect of Mobile Web Survey Design on Screen Orientation Manipulation
Year 2014
Access date 16.08.2016
Presentation PDF (1,33 MB)
Respondents are using mobile devices to complete web surveys whether we want them to or not. Because most web surveys are designed for larger screens, data quality may suffer, though little evidence exists to demonstrate whether or not this is the case. With on-board accelerometers, mobile devices can capture paradata to give us insight into respondent behavior.
To better evaluate whether specific web survey design features cause respondents to change how they hold their device we captured screen orientation, among other paradata. The assumption is that if the respondent rotates the device, it is to better view the survey. Capturing this information on a page-by-page basis throughout a survey provided sufficient data to understand if an orientation change (and change back) may have resulted from particular designs -- such as grid formatted questions.
We captured this paradata over multiple waves (of varying questionnaire length) to a college student survey fielded between mid-2013 and early 2014. While the survey was initially designed for larger screens, elements of mobile optimization were included for respondents accessing from a smartphone. In initially comparing optimized versus non-optimized designs, we see a significantly higher rate of respondents using landscape orientation on the non-optimized survey (approx. 20%) versus those on the optimized surveys (approx. 1%).
We will describe the respondent experience across multiple survey implementations, and we will discuss our hypotheses and further unveil the data that we have collected. Where possible, we will evaluate the data provided for quality (item missing data, breakoff, etc.) and will include device type and other interrelated variables in our analyses.
Year of publication2014
Bibliographic typeConferences, workshops, tutorials, presentations