In this paper we extend previous work on automatic forwarding (AF) versus manual forwarding (MF) to examine the effect on the cognitive response process. We expect respondent cognitive burden to increase as a combined function of low information accessibility and auto forwarding. We experimentally tested manual versus auto forwarding for varying levels information accessibility (low versus high) and need for consistency of responses (low versus high). We expect AF to perform better when information is readily accessible and the need to access previous questions (consistency) is low.

Methods & Data:

Keywords: response time, paradata

Undergraduate students at two universities in Finland were randomly assigned to six independent web survey conditions in two experiments (n=3028 and n=5004). The experimental design was an incomplete factorial, including three independent variables: forwarding procedure, information accessibility and consistency requirement. Total response times and elapsed times on individual items from paradata were analyzed using linear regression models and multilevel models. The returns and shifts between items as well as straight-line responding and consistent responding were examined.

Results:

Total response time was 27 seconds higher on average in the MF surveys. Less-accessible information and the consistency requirement both increased total response time (20 seconds and 14 seconds on average, respectively). All three main effects were statistically significant. Contrary to expectation, no significant interactions were found.

MF respondents were significantly more likely than AF respondents to change answers on experimentally-manipulated items. For example, 13 percent of manually forwarded group changed their responses on the question conveying less-accessible information compared with 2 - 4 % changes in the control groups. AF did not increase straight-line responding. However, based on experiments 1 and 2 we found slight evidence that AF enhances consistent responding when the need to access previous questions is high. AF respondents of experiment 2 self-reported responding easier than MF respondents (difference 10 %).

Added Value

There are many proponents of auto forwarding in web surveys. This paper provides one of the few carefully-designed and theoretically-motivated studies to explore this important design choice under experimentally-varied conditions. These results will help shape practice.