Survey non-response is one possible threat to generalizability of findings and has received considerable attention. Another possible threat to generalizability is item non-response. We were interested in developing an experimental test of various no-opinion options for evaluative tasks and comparing them with conditions which omit the no-opinion options.
Method Respondents: We had 9342 from the Harris Poll Online panel for an online survey. Procedure: Asked about grocery stores visited Assigned store to rate Rated familiarity, liking, recommendation, and likelihood to shop at store Assigned to rating format: 5 out-of-range response conditions - “Not applicable,” “Don’t know,” “Not sure,” “Not familiar with aspect,” “No opinion”, 1 non-mandatory condition, 2 forced response conditions Rated 10 aspects of store (e.g. pharmacy)
Results: The Not Applicable condition had the lowest endorsement of the out-of-range category, while the Don’t Know, Not Sure, Not Familiar, and No Opinion conditions were comparable in endorsement. Both forced response conditions were similar in their endorsement patterns, but were significantly different from the out-of-range response conditions. The forced response conditions had significantly fewer people who endorsed the top two boxes (“Excellent” and “Very Good”) but more people endorsed the bottom three boxes (“Somewhat good”, “Fair”, “Poor”). Correlational Patterns: We found that there were no significant differences between the conditions for the intercorrelations and correlations with the criteria.
Discussion: While the issue of item non-response is multi-faceted, our investigation suggests that the primary effects of allowing non-response is increased usage of the top categories of the scale with evaluative scales.