Notice: the WebSM website has not been updated since the beginning of 2018.

Web Survey Bibliography

Title Testing Web-Based Survey Measures of Gender Identity and Sexual Orientation Using Mark-All-That-Apply and Forced Choice Questions
Year 2016
Access date 03.06.2016
Best practices in measuring sexual orientation and gender identity suggest exhaustive but not necessarily exclusive response options. This goal implies the use of a mark-all-that-apply (MATA) to adequately measure these identities. However, problematic features of MATA can harm data quality. Consequently, forced choice — changing each response into a yes/no question — is typically preferred. While potentially increasing response validity, forced choice also increases respondent burden, especially on smartphones and for longer lists of response options. Thus, the trade-off between forced choice and mark-all-that-apply is between respondent burden on the one hand (which leads to increased breakoffs, partial interviews and unit nonresponse) and the potential for measurement error, on the other. But what if the topic is of high salience for the respondent? Highly motivated respondents are more likely to agree to participate and give higher quality responses, potentially overcoming somedesign problems. Thus, our research question is this: can respondents for whom the questions are salient overcome the flaws of the MATA? Building on pilot work, this survey experiment randomly assigns respondents from a web panel (with an LGBT oversample) into categories in a 2x3 factorial design. The first factor is question type: (1) MATA or (2) forced choice. The second factor is question presentation: (1) sexual orientation and gender identity in one question, (2) in two questions on one page; or (2) in two questions on two pages. Findings contribute to the body of knowledge on the self-identification practices of this specific population as well as methodological considerations emerging from satisficing in topics of high salience.
Year of publication2016
Bibliographic typeConferences, workshops, tutorials, presentations

Web survey bibliography - The American Association for Public Opinion Research (AAPOR) 71st Annual Conference, 2016 (107)