There are hundreds of studies which experimentally compare the effects of different modes of data collection on measurement. Some consistent findings and theories have emerged from these studies, notably regarding the measurement of sensitive attitudes and behaviours. Whether or not respondents will answer a particular survey question differently in one mode than in another is, nonetheless, still mostly unpredictable. This is in part because many experimental tests focus on descriptive comparisons of response distributions and do not attempt to identify generalisable features of the question or the modes that might cause the differences in measurement.

Based on existing theories and empirical findings, we develop a framework of the causes of mode effects on measurement, which can be used to assess the susceptibility of a survey question to mode effects. The framework posits three important sets of features, the first two of which often differ between modes: characteristics of the question design (e.g. the format), characteristics of the method of administration (e.g. whether or not an interviewer is present, whether information is transmitted to and from the respondent aurally or verbally), and more inherent features of a question, such as difficulty or sensitivity (though these features too may be mediated by question design and method of administration). To test predictions from our framework, we have carried out a mixed mode experiment with CAPI, CATI and web, followed by cognitive interviews to further understand the causes of differences in measurement caused by the use of different modes. This work is funded by the UK Economic and Social Research Council.