Relevance & Research Question: Matrix questions are and have always been a big problem in online fieldwork. On the one hand many statistical procedures suggest the use of matrix questions, whilst on the other hand quite a lot of data quality issues are related to matrix questions, - mainly due to satisficing behavior of respondents. One explanation for such a behavior is the high cognitive burden for the respondent when being confronted with the compact presentation of many options and a lot of text on little space. Steps towards a better data quality thus have to find smarter and more intuitive solutions for collecting the needed data. This contribution compares different alternatives for grid questions and shows how to use them in the right context.
Methods & Data: We conducted a study with five splits in total, each of them with a different alternative for the matrix question. Each split contained the respective alternative twice, one time in the context of brand likeability and one time in the context of general attitudes. These concepts have been compared by different metrics in regard to data quality (e.g. variance, time needed on question, consistency with answers in following questions), comparability of the results among these alternatives and the satisfaction of the respondents with usability of each question type. These differences are controlled by sociodemographic factors.
Results: Our results draw a differentiated picture for each of these question types. It seems that there is no universal solution that applies to all contexts. Nonetheless a bunch of recommendations can be drawn for the appropriate design of a questionnaire and the best practice for each of the presented question types.
Added Value: Especially with the growing importance of mobile research in the market research industry, an appropriate questionnaire design gains more and more importance. Smaller screens and new input devices (such as touchscreens) call for alternatives to conventional solutions during fieldwork. We show that researchers most often can count with more satisfied respondents and a better data quality when avoiding the standard matrix question type. Nonetheless they don’t have to do without valuable data.