A myriad of studies compare the estimates of identical questionnaires between different modes in order to determine patterns in survey measurement error. For example, some researchers compare estimates of socially undesirable behaviors between self-administered questionnaires like pen-and-paper or web and interviewer modes like telephone or face-to-face. Other studies compare proxies of satisficing. Such research influences our definitions of an “interviewer mode” and a “self-administered mode.”

This study examines how taken-for-granted characteristics of survey modes blur when self-administered web surveys become interactive. Using a 294-person survey experiment, we compare identical questionnaires between three modes: face-to-face interviewer, traditional web, and experimental web.

The experimental web condition uses a survey programmed to slow respondents down, re-question respondents who appear to answer with socially desirable responses, and probe respondents for more responses to open questions. This condition is then compared to an identical face-to-face survey that uses probes and re-questions respondents and a “traditional” web survey that does not use such interactions.

Experimental results show intriguing albeit mixed results. On certain questions, web respondents are less likely to answer in a socially desirable fashion than face-to-face respondents, regardless of the interactive nature of the web survey. For these questions, mode matters more than the interaction. For other questions, the interactive web mode and face-to-face mode provide identical response patterns, illustrating how interactions such as probes and requesting can lead to higher data quality. The same modal pattern emerges for issues of satisficing. These discrepancies in results illustrate how, as gender- and race-of-interviewer effects manifest only on specific questions, so do interactions on self-administered questionnaires matter for data quality on specific questions and sets of questions. The study concludes that web surveys should be viewed as more than static, self-administered questionnaires while acknowledging that survey mode boundaries are

blurry not discrete.