Web Survey Bibliography
It is universally acknowledged that the wording of a survey question can have a strong influence on the answers that respondents provide. For example, many studies have shown that vague and ambiguous terms are often interpreted idiosyncratically by respondents, and thus can increase measurement error. In addition to ambiguity, the cognitive effort required to comprehend survey questions may affect data quality in a similar way. This aspect of survey question design has received comparatively little attention to date and has rarely been examined experimentally. The present thesis suggests that applying a psycholinguistic perspective to survey question design may shed some light on the relationship between the cognitive effort required to comprehend survey questions and the quality of respondents’ answers. Theoretical and empirical evidence from psycholinguistics indicates that text (or question) comprehensibility is reduced by a set of specific text features: low-frequency words, vague or imprecise relative terms, vague or ambiguous noun phrases, complex syntax, complex logical structures, low syntactic redundancy, and bridging inferences. Three experimental studies were conducted to examine whether these seven text features indeed undermine question comprehensibility and, in turn, how question comprehensibility affects the quality of respondents’ answers. Study 1 revealed that six of the seven text features reduce question comprehensibility as indicated by significantly longer response times. Moreover, the text features were found to reduce response quality by producing more neutral (i.e., midpoint) answers. For the most part, these findings were supported by study 2, in which eye-tracking parameters were used as more direct measures of cognitive effort: respondents fixated longer on questions containing one of these text features and required more fixations to process, re-read and interpret these questions in comparison to control questions that did not include the text features. Finally, study 3 showed that respondents receiving less comprehensible questions provided lower-quality responses (as indicated by number of non-substantive responses, number of neutral responses, and over-time consistency of responses) than respondents receiving control questions that were easier to comprehend. Moreover, interaction effects of question comprehensibility with respondents’ verbal skills and their motivation to answer surveys were found. Taken together, these findings indicate that response quality is reduced if questions are difficult to comprehend and exceed the processing effort that respondents are willing or able to invest during survey responding. Hence, survey designers should try to minimize the cognitive effort required to comprehend a question by avoiding the problematic text features discussed above.
Universität Mannheim Homepage (abstract) / (full text)
Web Survey Bibliography (6476)
- Using Web Ex to Conduct Usability Testing of an On-Line Survey Instrument; 2013; Stettler, K.
- Effects of Lotteries on Response Behavior in Online Panels; 2013; Goeritz, A., Luthe, S. C.
- Lotteries and study results in market research online panels; 2013; Goeritz, A., Luthe, S. C.
- The Effects of Errors in Paradata on Weighting Class Adjustments: A Simulation Study; 2013; West, B. T.
- Using Paradata to Study Response to Within-Survey Requests; 2013; Sakshaug, J. W.
- Paradata for Coverage Research ; 2013; Eckman, S.
- Improving Surveys with Paradata: Analytic Uses of Process Information; 2013; Kreuter, F.
- Theory of adaptation or survival of the fittest?; 2013; Cavallaro, K.
- Online Fundraising Essentials, Second Edition; 2013; Stevenson, S. C.
- Tips for Evaluating Online Effectiveness; 2013; Stevenson, S. C.
- The Digital Divide: The internet and social inequality in international perspective; 2013; Ragnedda, M., Muschert, G.
- Ten questions to ask your online survey provider; 2013; Williams, D.
- Survey quality prediction system 2.0; 2013
- Practical tools for designing and weighting survey samples; 2013; Valliant, R. L., Daver, J. A., Kreuter, F.
- Paradata in web surveys; 2013; Callegaro, M.
- Report Of The AAPOR Task Force On Non-probability sampling; 2013; Baker, R. P., Brick, J. M., Bates, N., Battaglia, M. P., Couper, M. P., Dever, J. A., Gile, K. J., Tourangeau...
- Incentive effects; 2013; Goeritz, A.
- A nationwide web-based freight data collection; 2013; Samimi, A., Mohammadian, A., Kawamura, K.
- Mode Matters: Evaluating Response Comparability in a Mixed-Mode Survey; 2013; Bowyer, B. T., Rogowski, J. C.
- Comparing Survey Results Obtained via Mobile Devices and Computers: An Experiment With a Mobile Web...; 2013; de Bruijne, M., Wijnant, A.
- Cognitive Probes in Web Surveys: On the Effect of Different Text Box Size and Probing Exposure on Response...; 2013; Behr, D., Bandilla, W., Kaczmirek, L., Braun, M.
- The E-Interview in Qualitative Research; 2013; Bampton, R., Cowton, C., Downs, Y.
- Methodological Considerations of Qualitative Email Interviews; 2013; Nehls, K.
- Best Practice in Online Survey Research with Sensitive Topics; 2013; Kays, K., Keith, T. L., Broughal, M. T.
- Research Intentions are Nothing without Technology: Mixed-Method Web Surveys and the Coberen Wall of...; 2013; Ganassali, S., Rodriguez-Santos, C.
- Reducing Response Burden for Enterprises Combining Methods for Data Collection on the Internet; 2013; Vik, T.
- Measuring Wages Worldwide: Exploring the Potentials and Constraints of Volunteer Web Surveys; 2013; Steinmetz, S., Raess, D., Tijdens, K., de Pedraza, P.
- Using Web Surveys for Psychology Experiments: A Case Study in New Media Technology for Research; 2013; Peden, B. F., Tiry , A. M.
- The Distinctiveness of Online Research: Descriptive Assemblages, Unobtrusiveness, and Novel Kinds of...; 2013; Lanfrey, D.
- Sampling, Channels, and Contact Strategies in Internet Survey; 2013; Macrì, E., Tessitore, C.
- Advancing Research Methods with New Technologies; 2013; Sappleton, N.
- Data Quality in PC and Mobile Web Surveys; 2013; Mavletova, A. M.
- PDAs in socio-economic surveys: instrument bias, surveyor bias or both?; 2013; Escobal, J., Benites, S.
- Virtual research assistants: Replacing human interviewers by automated avatars in virtual worlds; 2013; Hasler, B. S., Tuchman, P., Friedman, D.
- Compared to a small, supervised lab experiment, a large, unsupervised web-based experiment on a previously...; 2013; Ryan, R. S., Wilde, M., Crist, S.
- From mixed-mode to multiple devices. Web surveys, smartphone surveys and apps: has the respondent gone...; 2013; Callegaro, M.
- Moving an established survey online – or not?; 2013; Barber, T., Chilvers, D., Kaul, S.
- Using mobile devices to access the realities of youth: How identification with society influences political...; 2013; Smith, M.
- On the Use of Latent Variable Models to Detect Differences in the Interpretation of Vague Quantifiers...; 2013; Griffin, J.
- Managing mobile research: How it's different and why it matters; 2013; Kachhi-Jiwani, D., Tucker, J., Wilding-Brown, L.
- An approach to selecting online respondents; 2013; Terhanian, G.
- By the Numbers: Theory of adaptation or survival of the fittest?; 2013; Cavallaro, K.
- Designing and conducting business surveys; 2013; Snijkers, G.,Araldsen, G., , Willimack, D. K.Jones, J.
- Battle of the Scales: Understanding Respondent Scale Usage in the US and Abroad; 2013; Courtright, M., Pashupati, K., Pettit, F. A.
- Modular Survey Design: A Bite Size Proposal; 2013; Kelly, F., Stevens, S., Johnson, A.
- Cyborgs vs. Monsters: Assembling Modular Surveys to Create Complete Datasets; 2013; Johnson, E. P., Siluk, L., Tarraf, S.
- Do I Have Your Full Attention?; 2013; Cape, P. J.
- Does Sample Size Still Matter?; 2013; Bakken, D. G., Bond, M.
- Optimizing Surveys for Smartphones: Maximizing Response Rates While Minimizing Bias; 2013; Lattery, K., Park Bartolone, G., Saunders, T.
- Shorter Isn't Always Better; 2013; Burdein, I.

