Web Survey Bibliography
Title A Computer Tool that Helps Survey Methodologists Improve the Comprehensibility of Questions
Author Graesser, A., Karnavat, A., Daniel, K. F., Cooper, E., Cai, Z., Whitten, S., Louwerse, M., Bartlett, K., Tenneti, R., Ratnakar, V., Ajzen, I.
Year 2002
Access date 01.06.2004
Abstract Survey methodologists need analytical tools that help them identify problems with questions with respect to comprehension difficulty. We developed a computer tool (called QUAID) that assists survey methodologists in improving the wording, syntax, and semantics of questions. QUAID stands for Question Understanding Aid. The input to QUAID consists of a question on a questionnaire, context information, and answer alternatives. The output is a list of potential problems with the question, including: unfamiliar technical terms, vague or imprecise relative terms, vague or ambiguous noun-phrases, complex syntax, and working memory overload. QUAID is currently available on a web facility (http://mnemosyne.csl.psyc.memphis.edu/quaid/quaidindex.html). We have evaluated how well QUAID diagnoses five categories of problems with questions. We analyzed 550 questions sampled from 11 surveys of the US Census Bureau. Experts in language and cognition rated each question as to whether it had the five problems; the expert ratings were compared with QUAID output. Analyses of hit rates, false alarm rates, and discrimination scores confirmed there is a significant correspondence between QUAID and the judgments of experts. Eye tracking data have also been collected from respondents as they answer questions. We explored whether the patterns of eye fixations can be predicted by particular problems with questions.
Year of publication2002
Bibliographic typeConferences, workshops, tutorials, presentations
