Web surveys are increasingly used for data collection in social science research since they offer several substantial benefits: cost-effectiveness, saving time, and most importantly, they enable researchers to capture a variety of paradata (e.g., response times). Web mode, however, might also support respondent’s distraction during survey completion due to "multi-tasking" (e.g., checking incoming emails, changing to other websites, or starting programs). Until now, it lacks empirical evidence in which specific way distraction during survey participation affects the response behavior of respondents.

Methods & Data:

In this study, we, therefore, investigate whether there are systematic differences between respondents who process the survey continuously and those do not. For this purpose, we use a new paradata tool called "SurveyFocus (SF)" – enabling survey researchers to gather the activity of the web-survey page. This cross-sectional study (n = 1,751) is based on an onomastic sampling approach and contained single as well as grid questions.

Results:

Our statistical analyses reveal substantial differences between continuously and discontinuously processing respondents. This implies that respondents who leave the web survey for a certain time produce significantly longer processing times (after correcting for the "time-out"). They additionally produce lower response quality in terms of item non-response and error of central tendency. Furthermore, there are considerable differences between single and grid questions.

Added Value:

All in all, our empirical findings suggest that the continuity of web-survey processing matters. For this reason, survey researchers and practitioners should take this circumstance into consideration when analyzing and interpreting web-survey data.