Web Survey Bibliography
It is universally acknowledged that the wording of a survey question can have a strong influence on the answers that respondents provide. For example, many studies have shown that vague and ambiguous terms are often interpreted idiosyncratically by respondents, and thus can increase measurement error. In addition to ambiguity, the cognitive effort required to comprehend survey questions may affect data quality in a similar way. This aspect of survey question design has received comparatively little attention to date and has rarely been examined experimentally. The present thesis suggests that applying a psycholinguistic perspective to survey question design may shed some light on the relationship between the cognitive effort required to comprehend survey questions and the quality of respondents’ answers. Theoretical and empirical evidence from psycholinguistics indicates that text (or question) comprehensibility is reduced by a set of specific text features: low-frequency words, vague or imprecise relative terms, vague or ambiguous noun phrases, complex syntax, complex logical structures, low syntactic redundancy, and bridging inferences. Three experimental studies were conducted to examine whether these seven text features indeed undermine question comprehensibility and, in turn, how question comprehensibility affects the quality of respondents’ answers. Study 1 revealed that six of the seven text features reduce question comprehensibility as indicated by significantly longer response times. Moreover, the text features were found to reduce response quality by producing more neutral (i.e., midpoint) answers. For the most part, these findings were supported by study 2, in which eye-tracking parameters were used as more direct measures of cognitive effort: respondents fixated longer on questions containing one of these text features and required more fixations to process, re-read and interpret these questions in comparison to control questions that did not include the text features. Finally, study 3 showed that respondents receiving less comprehensible questions provided lower-quality responses (as indicated by number of non-substantive responses, number of neutral responses, and over-time consistency of responses) than respondents receiving control questions that were easier to comprehend. Moreover, interaction effects of question comprehensibility with respondents’ verbal skills and their motivation to answer surveys were found. Taken together, these findings indicate that response quality is reduced if questions are difficult to comprehend and exceed the processing effort that respondents are willing or able to invest during survey responding. Hence, survey designers should try to minimize the cognitive effort required to comprehend a question by avoiding the problematic text features discussed above.
Universität Mannheim Homepage (abstract) / (full text)
Web Survey Bibliography (6390)
- Is „chapterisation“ a viable alternative to traditional progress indicators ?; 2012; Spicer, R., Dowling, Z.
- Internet use in households and by individual in 2012. Eurostat Statistics in Focus 50/2012; 2012; Seybert, H.
- Internet access - Households and individuals, 2012 part 2; 2012
- Internet access - Households and individuals, 2012; 2012
- Guide to social science data preparation. Best practice throughout the data life cycle; 2012
- Google et Médiamétrie créent une audience bimédia; 2012; Gonzales, P.
- GMI Pinnacle; 2012
- Global market research 2012; 2012
- Flowing with the mainstream. Is mobile market research finally living up to the hype?; 2012; Townsend, L.
- Explaining rising nonresponse rates in cross-sectional surveys; 2012; Brick, J. M., Williams, D.
- Eurobarometer Special surveys: Special Eurobarometer 381; 2012
- Online Surveys 2.0; 2012; Elferink, R.
- The Impact of Academic Sponsorship on Online Survey Dropout Rates; 2012; Allen, P. J., Roberts, L. D.
- Especially for You: Motivating Respondents in an Internet Panel by Offering Tailored Questions; 2012; Oudejans, M.
- Social media as a data collection tool: the impact of Facebook in behavioural research; 2012; Zoppos, E.
- Smartphone Apps and User Engagement: Collecting Data in the Digital Era; 2012; Link, M. W.
- Snowball Sampling in Online Social Networks; 2012; Raissi, M., Ackland, R.
- The Use of Facebook as a Locating and Contacting Tool; 2012; McCarthy, T.
- How Often Do You Use the App with a Bird on It? Exploring Differences in Survey Completion Times, Primacy...; 2012; Buskirk, T. D.
- Data quality of questions sensitive to social-desirability bias in web surveys; 2012; Lozar Manfreda, K., Zajc, N., Berzelak, N., Vehovar, V.
- Online Questionnaires: Development of ‘basic requirements’; 2012; Tries, S., Blanke, K.
- Social research in online context: methodological reflections on web surveys from a case study; 2012; Pandolfini, V.
- Efficacy of a health-related Facebook social network site on health-seeking behaviors; 2012; Woolley, P., Peterson, M.
- Methods for eliminating skip statements from questionnaire logic; 2012; Canvanough Spencer, S.
- The war against unengaged online respondents; 2012; Gittelman, S. H., Trimarchi, E.
- Qualitatively Speaking: The five absolute, no-excuse must-dos for online qualitative researchers; 2012; Rossow, A.
- By the Numbers: Lessons for using online panels in B2B research; 2012; Elsner, N.
- Improving Survey Website Usability ; 2012; Vannette, D.
- Specialized Tools for Measuring Past Events ; 2012; Belli, R. F.
- Transparency, Access and the Credibility of Survey Research; 2012; Lupia, A.
- Experience Sampling and Ecological Momentary Assessment; 2012; Stone, A.
- Can Microtargeting Improve Survey Sampling? An Assessment of Accuracy and Bias in Consumer File Marketing...; 2012; Pasek, J.
- Anonymity and Confidentiality; 2012; Tourangeau, R.
- Cognitive Evaluation of Survey Instruments: State of the Science (Art?) and Future Directions; 2012; Willis, G. B.
- Oh, Just One More Thing … Leveraging “Leave-Behinds” in Data Collection; 2012; Link, M. W.
- Can Official Records Correct Errors in Turnout Self-reports?; 2012; Berent, M., Krosnick, J. A., Lupia, A.
- Paradata; 2012; Kreuter, F.
- Computation of Survey Weights: Bridging Theory and Practice; 2012; Debell, M.
- Optimizing Response Rates; 2012; Brick, J. M.
- Modes of Data Collection; 2012; Tourangeau, R.
- The Use and Effects of Incentives in Surveys; 2012; Singer, E.
- Probability vs. Non-probability Methods; 2012; Langer, G,
- Improving Question Design to Maximize Reliability and Validity; 2012; Krosnick, J. A.
- Respondent Attrition vs Data Attrition and Their Reduction; 2012; Olsen, R. J.
- Survey Interviewing: Deviations from the Script; 2012; Schaeffer, N. C.
- Sampling for Single and Multi-Mode Surveys using Address-Based Sampling; 2012; O'Muircheartaigh, C.
- What Human Language Technology can do for you (and vice versa); 2012; Liberman, M.
- Proxy Reporting; 2012; Cobb, C. L.
- The Impact of Survery Nonresponse on Survey Accuracy; 2012; Keeter, S.
- How accurate are surveys of objective phenomena?; 2012; Chang, L. C., Krosnick, J. A.

