Web Survey Bibliography
In the literature on questionnaire design and survey methodology, pre-testing is mentioned as a way to evaluate questionnaires (i.e. investigate whether they work as intended) and control for measurement errors (i.e. assess data quality). As the American Statistical Association puts it (ASA, 1999, p. 11): “The questionnaire designer must understand the need to pretest, pretest, and then pretest some more.” Clark and Schober (1992, p. 29) indicate why this need to pre-test: “Surveyors cannot possibly write perfect questions, self-evident to each respondent, that never need clarification. And because they cannot, the answers will often be surprising.”
In this Ph.D. thesis I have tried to systematically describe my experiences with pre-test research at the Questionnaire Laboratory at Statistics Netherlands, a cognitive laboratory which started its work in 1992. This text is not aimed at a theoretical discussion of cognitive laboratory methods, but focuses on the application of these methods: setting-up and carrying-out pre-test research, analysing the data and presentation of the results.
The thesis starts with an introduction of cognitive laboratory research, including the history of the CASM (Cognitive Aspects of Survey Methodology) movement and the history of the Questionnaire Laboratory at Statistics Netherlands. The next two chapters address aspects of computer-assisted interviewing. Since at Statistics Netherlands most social-survey questionnaires are computerised, this sets the conditions for pre-test research at the Questionnaire Laboratory. Chapter 2 discusses computer-assisted interviewing in general; chapter 3 addresses the effects of computer-assisted interviewing on data quality. These three chapters are introductory to the chapters that follow, the actual core of the thesis, in which the application of cognitive laboratory methods, including several case studies, are discussed.
The methods used at the Questionnaire Laboratory at Statistics Netherlands are discussed in chapters 4 and 5. Chapter 4 presents an overview of pre-test methods. Expert appraisal, focus groups, in-depth interviewing (including follow-up probing, meaning-oriented probing, paraphrasing, targeted test questions, and vignettes), and behavioural coding are discussed from a practical point of view, i.e. how they are applied in the Questionnaire Laboratory. Computer-Assisted Qualitative Interviewing (CAQI) is discussed in chapter 5. The CAQI method has been developed at the Questionnaire Laboratory to pre-test computerised questionnaires. With CAQI a pre-test protocol is integrated in a computerised questionnaire to be tested.
The next four chapters present case studies of cognitive research in which the methods addressed in the chapter 4 and 5 have been applied. These chapters discuss the design and the results (i.e. identified problems in the questionnaire and recommendations for improvement) of these studies.
Chapter 10 concludes this thesis with a summary. A number of identified problems in the investigated questionnaires are: vague or unclear wording, complex syntax, long question, double-barrelled question, conflict with previous question(s), question asks for specific information that is not available by heart, difficult to come to an answer because of complex calculation, overlapping or missing response items. In the conclusion, the identified problems are related to design errors in the questionnaire.
igitur Homepage (abstract) / (full text)
Web survey bibliography - 2002 (87)
- On-line qualitative market research: Interviewing the world at a fingertip; 2002; Scholl, N., Mulders, S., Drent, R.
- A lesson model: Using the Web for research; 2002; Grant, S.
- Willingness to Pay for Organic Foods: A Comparison between Survey Data and Panel Data from Denmark ; 2002; Millock, K.; Hansen, L.G.; Wier, M.; Andersen, L.M
- The Differential Effects of Face-to-Face and Computer Interview Modes; 2002; Cooley, P. C.; Des Jarlais, D.C.; Gribble, J.; Newman, J.C.; F.; Paone, D.; Turner, C. F.
- Group decision making in a multiple criteria environment: A case using the AHP in software selection; 2002; Lai, V.S., Wong, B.K., Cheung, W.
- Cognitive Laboratory Experiences : On Pre-testing Computerised Questionnaires; 2002; Snijkers, G.
- Methodological Issues In Web-based Research; 2002; Duffy, M. E.
- Solicitation by e-Mail and solicitor's status: A field study of social research influence on the...; 2002; Gueguen, N., Jacob, C.
- Trends in marketing research and their impact on survey research sampling; 2002; Anich, B.
- Standards questionnaire format; 2002
- Psychological reactions to terrorist attacks. Findings from the national study of American's reactions...; 2002; Thalji, L., Schlenger, W. E., Caddell, J. M., Ebert, L., Jordan, B. K., Rourke, K. M., Willson, D.,...
- Predicting the future of consumer panels; 2002; Wansink, B., Sudman, S.
- New approaches to assessing opinion: The prospects for electronic mail surveys; 2002; Best, S. J., Krueger, B. S.
- More than a thousand words? Visual cues and visual knowledge; 2002; Prior, M.
- E-Research: Ethics, security, design, and control in psychological research on the Internet; 2002; Banaji, M., Greenwald, A., Nosek, B.
- Demand characteristics in the laboratory and the clinic: Conversations and collaborations with subjects...; 2002; Kihlstrom, J. F.
- Bracketing effects in categorized survey questions and the measurement of economic quantities; 2002; Winter, J.
- A D-minus for computer exams; 2002; Mayfield, K.
- The use of incentives to reduce nonresponse household surveys; 2002; Singer, E.
- Ethnical guidelines for internet research; 2002
- WEXTOR: A Web-based tool for generating and visualizing experimental designs and procedures; 2002; Reips, U.-D., Neuhaus, C.
- Studying hate crime with the Internet: What makes racist advocate racial violence?; 2002; Glaser, J., Dixit, J., Green, D. P.
- Cognitive processes when answering online questionnaires; 2002; Fuchs, M.
- Using client side paradata to evaluate respondent behavior in Web surveys; 2002; Heerwegh, D.
- Doec propensity score weighting work for Web Surveys; 2002; Forsman, G., Varedian, M.
- Visual design issues in Web surveys; 2002; Couper, M. P.
- Results from cognitive interviews of nsf earned doctorate Web survey; 2002; Altheimer, I., Dillman, D. A.
- Conducting Survey Research Through an Enhanced Online Web Survey Procedure ; 2002; Theuri, P. M., Turner, L. D.
- Community views on crime and policing: Survey mode effects on bias in community surveys; 2002; Hennigan, K. M., Maxson, C. L., Sloane, D., Ranney, M.
- From Paper-and-Pencil to Screen-and-Keyboard: An Empirical Assessment of Equivalence Issues in Internet...; 2002; Deutskens, E., de Ruyter, K., Wetzels, M.
- Survey of Doctors' Experience of Patients Using the Internet; 2002; Potts, H. W. W., Wyatt, J. C., Pagerey, P. D.
- Libraries and desktop storage options: results of a Web-based survey; 2002; Hendricks, A., Wang, Ji.
- The local/global decision in Internet research; 2002; Crawford, S. D., Jones, S., Rowen, C.
- Hard Choices, Easy Answers: Values, Information, and American Public Opinion; 2002; Alvarez, M. R., Brehm, J.
- Evaluation of Web Survey Data Collection Systems; 2002; Crawford, S. D.
- Online surveys with simple random samples for large populations; 2002; Kreuels, B.
- Sampling problems in research on personal Homepages; 2002; Marcus, B., Machilek, F., Schuetz, A., Kilian, C.
- Selecting a consumer panel service; 2002; Sudman, S., Wansink, B.
- Compare and contrast; 2002; Kindig, L.
- Software Review: CAVI from OpinionOne; 2002; Macer, T.
- Supplier Side: Survey programming integration; 2002; Mitrano, M.
- Humbug, science, survey research; 2002; Litman, J.
- Drawing a few from many; 2002; Fitzgerald, A.
- If given the choice; 2002; Allison, J., O'Konis, C.
- At my own pace in my own place; 2002; Downes-Le Guin, T.
- Privacy, please; 2002; Mack, B.
- My opinion counts; 2002; Solomon, M. B.
- Multi-mode research dos and don'ts; 2002; Hogg, A.
- Qualitatively Speaking: Online focus groups...here today, not gone tomorrow; 2002; Yoffie, A. J.
- Net research suffers due to failed promise; 2002; Tortorello, N. J.