Relevance & Research Question
When designing questionnaires, an important decision to make is whether or not to include a ‘do-not-know-option’. In interviews this dilemma is solved by not explicitly offering ‘do-not-know’, but accepting it when it occurs. Interviewers are instructed to accept a non-substantive answer only after a gentle probe.
Online surveys, being self-administered, lack an interviewer. Therefore, web survey designers are hesitant to offer an explicit do-not-know option and ‘required answer’ is often default standard software. However survey methodologist strongly advice against this forced-answer strategy. Requiring an answer does not necessarily ensure that the right answer is given and may lead to irritation and more break-offs or to guessing and less valid answers, thereby reducing data quality.
Methods and Data
The data were collected among members of the LISS panel, a probability based panel of the Dutch population. The questionnaire contained questions, which in previous self-administered surveys showed a high percentage of item-nonresponse. A three by two experimental design was used. Factor A manipulated no explicit offering vs offering do-not-know in two different ways, visually separating do-not know and offering do-not know as a special button. Factor B manipulated accepting a do-not-know vs. only accepting it after a friendly probe. Respondents were randomly assigned to experimental conditions.
Results
We found clear effects of offering ‘do-not-know’ and of probing. Not explicitly offering do-not-know (but allowing to skip) followed by a friendly probe resulted in the lowest amount of missing information. Respondent evaluations showed that when do-not-know was offered explicitly the questions were experienced as less difficult. When a probe was offered, respondents indicated that the questions made them think more (about the topic). These results suggest that offering a d-not-know without probing gives respondents an easy escape, while probing stimulates the question-answer process. The scale reliabilities support this.
Added Value
This study adds an empirical basis to the debate on whether or not to offer do-not-know options in web surveys. We show that explicitly offering a do-not-know option in a web survey is not advisable. Allowing respondents to skip a question and programming in friendly probes is a good alternative.