Notice: the WebSM website has not been updated since the beginning of 2018.

Web Survey Bibliography

Title Using eye-tracking to understand how fourth grade students answer matrix items
Year 2015
Access date 23.08.2016
Full text PDF (522 KB)
Abstract
Eye-tracking has been used to better understand the survey response process. For instance, eye-tracking has been used to identify questions that are difficult to comprehend, how to present long lists of response options, and to measure the length of fixation on definitions in Web surveys. Matrix questions have been commonly used in Web surveys as well as in other types of surveys.The literature shows respondents took less time to answer questions when they were presented in a matrix than when they were presented individually across separate pages or screens. The use of matrix questions, however, may also be associated with several undesirable outcomes, including higher breakoff rates, higher missing data rates, and straightlining. Relatively little is known about how children answer matrix questions. This paper demonstrates how eye-tracking was used to determine the feasibility of using matrix questions to measure the background characteristics of students in the National Assessment of Educational Progress (NAEP) questionnaire. Fourth grade students were answered both matrix and discrete versions of questions on tablet computers while wearing real-world eye tracking glasses. This study addresses four research questions related to the use of eye-tracking to test survey questions. First, we examine whether matrix items require more effort to answer than discrete items for fourth grade students. Second, we investigate how the processing of sub items change within a matrix. Third, we examine how the processing of questions change over time. In order to address these research questions, we examine difference in the mean number of fixations per word and the mean duration per word for matrix and discrete questions. Overall, the study finds support for the use of matrix questions with fourth grade students in the NAEP. Implications for the use of eye-tracking equipment to evaluate survey questions are also discussed.
Year of publication2015
Bibliographic typeConferences, workshops, tutorials, presentations
Print