When they fill out a questionnaire, the respondents mobilize their experience on the theme of the survey but they also take position towards the questionnaire. Understanding how a questionnaire is perceived by respondents is thus necessary in order to improve the survey methodology and to better appreciate the collected data. Open-ended questions are an appropriate way to have a better image of the understanding, the impact and the relevance of the questionnaire.
The paper will take the example of the ELVIRE survey, studying the languages used nationally and internationally by researchers working in the French public research field. ELVIRE targeted all researchers and Ph.D. students working in all the public research units. Almost 8,900 respondents filled out the self-administered Web questionnaire.
This questionnaire included large place for free comments and finished on an open-ended question asking the opinion of the respondents on the questionnaire. This provides an extensive corpus of comments, thoughts and judgements on the survey.
We will compare different approaches: post-coding and multidimensional textual statistics with comparisons with the distribution of words. We will thus investigate what this kind of open-ended questions, analysed in relation to the socio-demographic characteristics of the respondents, could teach us about their perception of: – The topics broached by the survey and the questions’ relevance; – The ideological position and the objectives of the survey; – The design and the implementation of the questionnaire.