Notice: the WebSM website has not been updated since the beginning of 2018.

Web Survey Bibliography

Title Investigating Measurement Error through Survey Question Placement
Year 2016
Access date 09.06.2016
Previous designs of a national, cross-sectional survey on how students and their families pay for postsecondary education have focused on measuring and reducing nonresponse bias. However, the potential for measurement error in the survey estimates remain unstudied. This paper presents the results of several experiments aimed at investigating measurement error in this comprehensive student survey. We look at two indicators of measurement error – differences in responses to questions as a function of their placement in the survey (early vs. late) and response accuracy as a function of question placement in the survey. For differences in responses as a function of question placement, we examine responses to fictitious student loan questions and reverse coded questions placed in different parts of the survey. In order to examine response accuracy, we compare responses to federal student loan questions placed early vs. late in the survey, with actual loan amounts available through an administrative data matching source. Overall, about 10% of respondents provided substantive responses to fictitious questions. Contrary to expectations, the percentage was significantly higher when the fictitious item was placed in the first half of the survey. In addition, a significantly higher percent of substantive responses was obtained when the interview was conducted by phone rather than self-administered (web or mobile web). Similarly, about 9% of respondents provided opposing opinions when answering a reverse-worded item later in the survey. Finally, respondents tended to overestimate their loan amounts regardless of where in the survey the item was placed – we found more reporting error to questions asked earlier rather than later in the survey, but the difference did not reach statistical significance. We explore the potential implications of these results for current and future survey designs.
Year of publication2016
Bibliographic typeConferences, workshops, tutorials, presentations

Web survey bibliography - The American Association for Public Opinion Research (AAPOR) 71st Annual Conference, 2016 (107)