Web Survey Bibliography
It is universally acknowledged that the wording of a survey question can have a strong influence on the answers that respondents provide. For example, many studies have shown that vague and ambiguous terms are often interpreted idiosyncratically by respondents, and thus can increase measurement error. In addition to ambiguity, the cognitive effort required to comprehend survey questions may affect data quality in a similar way. This aspect of survey question design has received comparatively little attention to date and has rarely been examined experimentally. The present thesis suggests that applying a psycholinguistic perspective to survey question design may shed some light on the relationship between the cognitive effort required to comprehend survey questions and the quality of respondents’ answers. Theoretical and empirical evidence from psycholinguistics indicates that text (or question) comprehensibility is reduced by a set of specific text features: low-frequency words, vague or imprecise relative terms, vague or ambiguous noun phrases, complex syntax, complex logical structures, low syntactic redundancy, and bridging inferences. Three experimental studies were conducted to examine whether these seven text features indeed undermine question comprehensibility and, in turn, how question comprehensibility affects the quality of respondents’ answers. Study 1 revealed that six of the seven text features reduce question comprehensibility as indicated by significantly longer response times. Moreover, the text features were found to reduce response quality by producing more neutral (i.e., midpoint) answers. For the most part, these findings were supported by study 2, in which eye-tracking parameters were used as more direct measures of cognitive effort: respondents fixated longer on questions containing one of these text features and required more fixations to process, re-read and interpret these questions in comparison to control questions that did not include the text features. Finally, study 3 showed that respondents receiving less comprehensible questions provided lower-quality responses (as indicated by number of non-substantive responses, number of neutral responses, and over-time consistency of responses) than respondents receiving control questions that were easier to comprehend. Moreover, interaction effects of question comprehensibility with respondents’ verbal skills and their motivation to answer surveys were found. Taken together, these findings indicate that response quality is reduced if questions are difficult to comprehend and exceed the processing effort that respondents are willing or able to invest during survey responding. Hence, survey designers should try to minimize the cognitive effort required to comprehend a question by avoiding the problematic text features discussed above.
Universität Mannheim Homepage (abstract) / (full text)
Web Survey Bibliography - Germany (420)
- Online Survey Participation via Mobile Devices: Findings From Seven Access Panel Studies; 2013; Bosnjak, M., Poggio, T., Funke, F.
- Use of Drag-and-Drop Rating Scales in Web Surveys and Its Effect on Survey Reports and Data Quality; 2013; Kunz, T.
- Online Panels: Recruitment Based on “Hot Topics” – What are the Consequences?; 2013; Andreasson, M., Martinsson, J.
- The Effectiveness of Mailed Invitations for Web Surveys; 2013; Bandilla, W., Couper, M. P., Kaczmirek, L.
- Effects of Lotteries on Response Behavior in Online Panels; 2013; Goeritz, A., Luthe, S. C.
- Lotteries and study results in market research online panels; 2013; Goeritz, A., Luthe, S. C.
- Cognitive Probes in Web Surveys: On the Effect of Different Text Box Size and Probing Exposure on Response...; 2013; Behr, D., Bandilla, W., Kaczmirek, L., Braun, M.
- Pros and cons of virtual interviewers – vote in the discussion about surveytainment; 2013; Póltorak, M., Kowalski, J.
- The fish model: What factors affect participants while filling in an online questionnaire?; 2013; Mohamed, B., Lorenz, A., Pscheida, D.
- Interview Duration in Web Surveys: Integrating Different Levels of Explanation; 2013; Rossmann, J., Gummer, T.
- The monetary value of good questionnaire design; 2013; Tress, F.
- Technical and methodological meta-information on current practices in online research: A full population...; 2013; Burger, C., Stieger, S.
- Using interactive feedback to enhance response quality in Web surveys. The case of open-ended questions...; 2013; Emde, M., Fuchs, M.
- Reducing Response Order Effects in Check-All-That-Apply Questions by Use of Dynamic Tooltip Instructions...; 2013; Kunz, T., Fuchs, M.
- Slide to ruin data: How slider scales may negatively affect data quality and what to do about it; 2013; Funke, F.
- Measuring wages via a volunteer web survey – a cross-national analysis of item nonresponse; 2013; Steinmetz, S., Annmaria, B.
- Identifying and Mitigating Satisficing in Web Surveys: Some Experimental Evidence; 2013; Blumenstiel, J. E., Rossmann, J.
- Does one really know?: Avoiding noninformative answers in a reliable way.; 2013; de Leeuw, E. D., Boevee, A., Hox, J.
- Online Mixed Mode Surveying using a Responsive Design; 2013; Kissau, K.
- Sensitive Topics in PC and Mobile Web Surveys; 2013; Mavletova, A. M., Couper, M. P.
- Mobile Research Performance: How Mobile Respondents Differ from PC Users Concerning Interview Quality...; 2013; Schmidt, S., Wenzel, O.
- Who responds to website visitor satisfaction surveys?; 2013; Andreadis, I.
- Measuring working conditions in a volunteer web survey; 2013; de Pedraza, P., Villacampa, A.
- Sampling online communities: using triplets as basis for a (semi-) automated hyperlink web crawler.; 2013; Veny, Y.
- Prison break: Releasing offline experiments from methodological constraints by transforming them into...; 2013; Förstel, H., Manthei, K., Mohnen, A., Berger, G.
- Comparison of psychometric properties of internet versions of the Marlowe-Crowne Social Desirability...; 2013; Vesteinsdottir, V., Reips, U. -D., Joinson, A. N., Porsdottir, F.
- Why are you leaving me?? - Personality predictors of answering drop out in an online-study; 2013; Thielsch, M., Nestler, S., Back, M.
- Propensity Score Weighting – Can Personality Adjust for Selectivity?; 2013; Glantz, A., Greszki, R.
- Research Design as an Influencing Factor for Reliability in Online Market Research; 2013; Wengrzik, J., Theuner, G.
- Ethics, privacy and data security in web-based course evaluation; 2013; Salaschek, M., Meese, C., Thielsch, M.
- Seducing the respondent – how to optimise invitations in on-site online research?; 2013; Póltorak, M., Kowalski, J.
- Influence of mobile devices in online surveys; 2013; Maxl, E., Baumgartner, T.
- E-questionnaire in cross-sectional household surveys; 2013; Karaganis, M.
- GESIS Online Panel Pilot: Results from a Probability-Based Online Access Panel; 2013; Kaczmirek, L., Bandilla, W., Schaurer, I., Struminskaya, B., Weyandt, K.
- Online Survey – Research with children on advertising impact; 2013; Funkenweh, V., Busch, J., Amthor, A. L., Boeer, A., Gaedke, J.
- HTML5 and mobile Web surveys: A Web experiment on new input types; 2013; Funke, F.
- Metadata on the demographics of online research: Results from a full-range study of available online...; 2013; Burger, C., Stieger, S.
- How the screen-out influence the dropout of a commercial panel; 2013; Bartoli, B.
- Beyond methodology - some ethical implications of "doing research online"; 2013; Heise, N.
- Innovation in Data Collection: the Responsive Design Approach; 2013; Bianchi, A., Biffignandi, S.
- Break-off and attrition in the GIP amongst technologically experienced and inexperienced participants...; 2013; Blom, A. G., Bossert, D., Clark, V., Funke, F., Gebhard, F., Holthausen, A., Krieger, U., Wachenfeld...
- Nonresponse and Nonresponse Bias in a Probability-Based Internet Panel; 2013; Blom, A. G., Bossert, D., Funke, F., Gebhard, F., Holthausen, A., Krieger, U.
- Rewards - Money for Nothing?; 2013; Cape, P. J., Martin, P.
- Effects of incentive reduction after a series of higher incentive waves in a probability-based online...; 2013; Struminskaya, B., Kaczmirek, L., Schaurer, I., Bandilla, W.
- Timing of Nonparticipation in an Online Panel: The effect of incentive strategies; 2013; Douhou, S., Scherpenzeel, A.
- How Do Lotteries and Study Results Influence Response Behavior in Online Panels?; 2013; Goeritz, A., Luthe, S. C.
- Sample composition discrepancies in different stages of a probability-based online panel; 2013; Bosnjak, M., Haas, I., Galesic, M., Kaczmirek, L., Bandilla, W., Couper, M. P.
- Metering mobile usage. Insights from global Arbitron mobile trends panel; 2012; Verkasalo, H.
- Is „chapterisation“ a viable alternative to traditional progress indicators ?; 2012; Spicer, R., Dowling, Z.
- Online Questionnaires: Development of ‘basic requirements’; 2012; Tries, S., Blanke, K.
