Respondents are more likely to disagree with negative questions (This article is bad. Yes/No), than to agree with positive ones (This article is good. Yes/No). Taking a cognitive perspective, two possible causes can be distinguished for this answering difference. One: positive and negative questions measure different attitudes, and therefore differ in their validity. Two: contrastive questions measure the same underlying attitude, but this attitude is expressed differently to the response scale because of the wording: even though response options like yes and no are straight opposites, the answer yes to a positive question is not equivalent to a no-answer to a negative question. This implies that contrastive questions are equally valid.

In the current study, we used eye-tracking to distinguish between these possibilities. We will reflect on the pros and cons of this type of paradata for investigating cognitive processes and data quality (question validity). Eye-tracking allows for an unobtrusive measure of cognitive processes and gives precise measures of where cognitive energy is spent. However, to relate eye-tracking measures to cognitive processes, a clearly defined cognitive model is required. We used a simplified version of the “Tourangeau-model” to this end. Yet, in lack of pre-defined guidelines, an interpretation of eye-tracking data in terms of this model is not clear-cut. We will present the interpretation of our data and we will argue that probably positive and negative questions are equally valid: the comprehension of contrastive questions takes equally long, whereas fitting an opinion to the response options takes more time for negative questions.