Notice: the WebSM website has not been updated since the beginning of 2018.

Web Survey Bibliography

Title Estimation of Expected Academic Engagement Behaviors: The Use of Vague Quantifiers Versus Tallied Responses
Author Cole, J.
Year 2013
Access date 31.05.2013

This study sheds light on a rarely explored topic in survey research: do different behavior estimation procedures for past and expected behaviors produce different results? This study is based on prior research regarding the importance of academic expectations, estimation of behavior frequency (e.g., Schaeffer & Presser, 2003), and the use of vague quantifiers in survey research (e.g., Wright, Gaskell, O’Muircheartaigh, 1994).Data for this study are from the 2010 administration of the Beginning College Survey of Student Engagement. Responses from more than 28,000 first-year students enrolled at 68 institutions were included in this analysis. Items from the core survey were repeated at the end of the Web version of the survey. Respondents were reminded of their original response to the item (core survey items are presented with vague quantifiers: very often, often, sometimes, and never) and were then asked to again estimate their behavior by tallying or counting their behaviors. One of the general findings is that the magnitude (effect size) of the differences for the vague estimations was much larger than for the tallied estimations. This means that those doing “gap analysis” where data are used to identify areas where student expectations are not met, may want to consider if the results are more of an artifact of the response set, then any real difference in behavior frequency. This study also found that tallied estimates associated with vague quantifiers are not necessarily stable. For instance, in high school “very often” asking questions in class corresponded with a tallied count of this activity of 23 times per week. However, “very often” expecting to ask questions in class during their first year of college corresponded with a mean of 16 times per week (dpooled=.550). Full results will be presented and implications for survey research discussed.

Access/Direct link

Conference Homepage (abstract)

Bibliographic typeConferences, workshops, tutorials, presentations

Web survey bibliography (4086)