In online studies, assessing the occurrence of events has commonly used: 1) yes-no grid (YNG) (yes or no if the event has occurred) or 2) multiple response list (MRL) (indicate all that apply).
Experiment1: Online (N=876) ran parallel to phone survey (N=1018)(topic: 7 elements measuring protest behavior regarding companies). Online randomly assigned to either YNG or MRL.
Experiment2: Online only (N=1953) (topic: use and purchase of 10 food products). Randomly assigned to YNG, MRL, or combined use-purchase grid (CUPG). Randomly assigned 5 or 10 elements to indicate purchase and use.
Experiment3: Online (N=1471), phone (N=797) (topic: purchase behavior at convenience stores). Online randomly assigned to YNG or MRL and either 5, 10, 15, or 20 elements.
Results:
In Experiment 1, YNG had significantly higher incidence than the MRL (on average 9.38% higher). The MRL was closer to phone data than the YNG (average deviation of 4.23% versus 12.23%).
In Experiment 2, the incidence of purchase and use was high across all response formats. The YNG had a higher incidence (3.53% for use and 4.18% for purchase). The CUPG appeared to yield data most similar to the MRL. Five versus 10 elements did not significantly affect incidence.
In Experiment 3, the incidence of purchase with YNG was significantly higher than MRL (average of 9.92%). The YNG appeared to somewhat closer to the phone data than the MRL (average deviation of 7.58% versus 9.93%). Some decline in incidence was demonstrated with 15 and 20 elements for both YNG and MRL.
Discussion:
The YNG consistently led to higher levels of endorsement than the MRL. Which method would yield results similar to phone data is yet to be determined.