Web Survey Bibliography
Previous research shows that changes in number-label association can affect respondents’ answers. In our first study, conducted with members of KnowledgePanel®, we manipulated the order of presentation of a fully labeled five point satisfaction scale (very satisfied to very unsatisfied); the association of numbers from 1 to 5 to each scale point; and we also reversed the number order (from 5 to 1). This manipulation created six experimental conditions. For each condition, we presented seven different items on separate screens. The completion rate for this study, done in April 2008, was 71%.
In the second study we used only a polar point satisfaction scale and manipulated the order of the scale as well the number-label associations (1 to 5 or 5 to 1). This created 4 experimental conditions to which KnowledgePanel members were randomly assigned during a survey conducted in August of 2008 with a completion rate of 72%. For each condition, we presented seven different items on separate screens.
In the first study we investigated the differences between each version, focusing specifically on the following comparisons: response options with numbers associated vs. options without numbers; order of the response option presentation; and order of the number association. Results from experiment 1 suggest that (1) reversing the order of presentation makes a significant difference resulting in primacy effects and in an increase of time latency when the scale is shown starting from “Very unsatisfied”. (2) Changing the number-label association does not make much of a difference; it appears that for a fully labeled scale the label is almost overriding the number. (3) Respondents are, however, slightly confused when the number-label association does not go in the expected direction.
In the second study we compared the 4 experimental groups, first between (same order of presentation but reversing the number-label association), and then within (same number-label association but reversing the order of presentation of the scale). We did not find many significant differences among groups, thus leading us to conclude that for a polar point scale the number-label association has less of an impact than for a fully labeled scale.
Homepage (abstract)/(full text)
Web survey bibliography - Kruse, Y. (5)
- The Effect of Email Invitation Subject Title and Text on Online Survey Completion Rates in Internet...; 2009; Kruse, Y., Thomas, M., Nukulkij, P., Callegaro, M.
- The impact of news reports of survey findings on respondents in a longitudinal panel study; 2009; Tompson, T., Dennis, J. M., Kruse, Y.
- The Effect of Email Invitation Customization on Survey Completion Rates in an Internet Panel: A Meta...; 2009; Callegaro, M., Kruse, Y., Thomas, M., Nukulkij, P.
- Panel Conditioning and Attrition in the AP-Yahoo! News Election Panel Study; 2009; Kruse, Y., Callegaro, M., Dennis, J. M., DiSogra, C., Subias, S., Lawrence, M., Tompson, T.
- Effects of Pre-coding Response Options for Five Point Satisfaction Scale in Web Surveys; 2008; Callegaro, M., Wells, T., Kruse, Y.