Web Survey Bibliography
The difficulty of a survey question is known to be a serious source of response error (Bless et al., 1992; Knäuper et al., 1997; Krosnick, 1991). If questions are difficult to understand respondents are likely to give incorrect answers, they may become frustrated, they may guess, or they may even refuse to answer any further question. An important goal in questionnaire design is to write survey questions that are easy to understand and to answer. However, very little is known about the factors that determine question difficulty. Only recently have survey researchers started to look at specific text features in order to explain why some questions are easier to comprehend or have less cognitive burden than others (Lessler&Forsyth, 1996; Graesser et al., 2006; Tourangeau, 2000). Theoretical and empirical evidence from psycholinguistics suggests that these features (e.g. low-frequency words, vague nounphrases, left-embedded syntax) cause comprehension difficulties and can thus have a strong impact on survey response quality. In order to examine the effects of these text features, we conducted an online experiment in which two versions of the same question were compared using response latency times (Draisma&Dijkstra, 2004). One group (N=45) received well-formulated survey questions, the other group (N=47) answered questions which were suboptimal with respect to several psycholinguistic text features. The results show that the text features influenced the cognitive effort required to answer survey questions. This was tested with an ANCOVA (with reading rate as a covariate) using overall response latency time as the dependant variable. It revealed a significant difference between both conditions (p=0.02) with subjects answering the suboptimal questions having longer overall response latency times. Moreover, question difficulty affected the distribution of answers Hence, questionnaire designers are advised to pay attention to these text features when crafting questions. The experiment is currently replicated with a larger sample and a slightly modified questionnaire. Besides response latencies and answer distributions, also dropout will be analyzed and presented.
Web Survey Bibliography - 7th International Conference on Social Science Methodology (27)
- Use of a website to evaluate quality of work-life and quality of life among community workers helping...; 2008; Dupuis, G.
- Comparing the network structures in teams among companies: Extensions of the MTML approach; 2008; Agneessens, F., Contractor, N.
- Online poll in study of incentives to participation of youth in innovative development; 2008; Gvozdeva, E. S.
- Measuring ties on online forums; 2008; Ziberna, A., Vehovar, V., Jakulin, A.
- Using Internet Pages of Organisations as Data Source for Social Science Research; 2008; Baumgarten, B., Grauel, J.
- Social Web Data as a Source for Social Science Research. The Example of a German Online Dating Website...; 2008; Skopek, J., Schmitz, A.
- DHS CAPI Data Collection Model Using PDAs; 2008; Rojas, G.
- The relative Coverage Error in Telephone Surveys caused by Mobile-Only Populations across Europe; 2008; Fuchs, M.
- Chaotic Web: The challenge of Misinformation and Disinformation; 2008; Keshavarz, H.
- Mixed methods in online evaluation: benefits and problems; 2008; Kuckartz, U.
- Use of the Internet as a data collection tool: a methodological investigation of online synchronous...; 2008; Evans, A. R., Elford, J., Wiggins, D.
- Parallel Phone and Web-based Interviews: Effects of Sample and Weighting on Comparability and Validity...; 2008; Thomas, R. K., Krane, D., Taylor, H., Terhanian, G.
- Modern Telephony, the Web and Survey Management; 2008; Olsen, R. J.
- Mobile Web Survey Design; 2008; Peytchev, A. Hill, C.
- Impact of new technologies in data collection methods; 2008; Callegaro, M.
- Integration of the web into survey data collection: Balancing costs and errors; 2008; Vehovar, V., Berzelak, N., Lozar Manfreda, K.
- Assessing Semantic Differentials with Visual Analogue Scales in Web Surveys; 2008; Funke, F., Reips, U. -D.
- Response Non-Differentiation and Response Styles in Web-Based Studies: Causes and Consequences ; 2008; Frisina, L. T., Thomas, R. K.
- Communicative Channels, Cognitive Processes and Question Understanding: Results from a Randomized Field...; 2008; Fuchs, M.
- Psycholinguistic Determinants of Question Difficulty: A Web Experiment; 2008; Faa▀, T., Kaczmirek, L.
- Interactive Interventions in Web Surveys Can Improve Data Quality; 2008; Conrad, F. G., Tourangeau, R., P., Kennedy, C.Couper, M. P.
- Understanding the decision to participate in a survey: motives for refusal and preferred forms of contact...; 2008; ┴lvarez, R. M., Sevillano, V.
- Visual Analogue Scales in Cross Cultural Web Surveys ; 2008; Funke, F., Reips, U. -D., Thomas, R. K.
- Survey mode effects in smoking status assessment; 2008; Burns, E., Levinson, A.
- Uses of self-anchoring scales in web surveys; 2008; Van Acker, F., Theuns, P.
- Open-ended questions and online surveys: the mode effect in relation to length; 2008; Denscombe, M.
- New survey strategies for radio: the RFM’s “Ouvinte Mais” case study; 2008; Cordeiro, P.