Web Survey Bibliography
Title Comparing Web Surveys and Paper & Pencil Surveys: Measurement Error and Mode Differences
Author Fuchs, M.
Year 2002
Access date 21.04.2004
Abstract Based on the increasing methodological research web surveys are increasingly adopted by survey researchers. In terms of the validity of web surveys early studies suggested that--controlling for other sources of survey error--little differences are to be expected in terms of response distributions. However, little insight is available regarding the cognitive processes at work in a respondent's mind when answering a survey question on the internet. We assume that the cognitive capacity available when answering a survey question is impaired by the respondent's navigational efforts when working though the questionnaire. This would harm data quality. In order to assess the cognitive processing of a web survey we use well documented question order and response order experiments from the literature and administer them both under web survey conditions as well as under paper and pencil conditions. We aim to demonstrate that it is not sufficient to compare response distributions when evaluating differences of web surveys and paper and pencil surveys. In this talk we present results from a large scale field experiment (n = 5,000) on measurement error in a mixed mode survey (p & p and online) among the German youth conducted in spring of 2001. In this survey, 6 experiments on measurement error were incorporated: 2 experiments on question order, 2 experiments on response categories, and 2 experiments on response order in long lists.
Abstract - optional Nachdem die Methodenforschung sich relativ gründlich mit einzelnen Problemen und Aspekten von Online-Umfragen beschäftigt hat, werden sie mittlerweile in großem Stil in der Umfrageforschung eingesetzt. Erste vorliegende Vergleiche haben gezeigt, dass die online gewonnenen Daten sich nicht dramatisch von offline gewonnenen Ergebnissen unterscheiden. Weniger erforscht ist bisher jedoch, ob die kognitiven Prozesse des Befragten bei Beantworten einer Fragebogen-Frage im Internet mit denen vergleichbar sind, die bei der Beantwortung eines Papier-Fragebogens auftraten. Wir vermuten, dass die kognitive Kapazität des Befragten bei der Bearbeitung eines Online-Fragebogens durch die Navigation und andere Aspekte belastet werden, was die Datenqualität beeinträchtigen würde. Um die kognitiven Prozesse bei der Beantwortung der Fragebogen-Fragen zu untersuchen haben wir in der Literatur gut dokumentierte Experimente zur Fragereihenfolge und zur Antwortreihenfolge sowohl in einer Online-Befragung wie auch in einer schriftlichen Befragung administriert. Die Ergebnisse zeigen, dass die kognitiven Prozesse beim Befragten bei Online-Umfragen variieren und dass es zur Evaluation der Validität von online gewonnenen Daten nicht ausreicht Antwortverteilungen von Web-Umfragen und schriftlichen Befragungen zu vergleichen. Im Vortrag werden Ergebnisse eines groß angelegten Feld-Experimentes (n= 5.000) zu Messfehlern in Multi-Mode-Befragungen (Paper & Pencil und Online) referiert, die im Frühjahr 2001 unter deutschen Jugendlichen durchgeführt wurde. In diese Umfrage wurden 6 Experimente zur Fragereihenfolge, zur Antwortreihenfolge und zu Skalen-Effekten eingebaut, deren Ergebnisse vergleichend referiert werden sollen.
Access/Direct link Homepage - conference (abstract)
Year of publication2002
Bibliographic typeConferences, workshops, tutorials, presentations
Web Survey Bibliography - Psychological issues (234)
- A Comparison of Ranking versus Rating in Online Surveys; 2004; Thomas, R. K.
- Researching shyness: a contradiction in terms?; 2004; Scott, S.
- Attachment Style and Subjective Motivations for Sex; 2004; Davis, D., Shaver, P. R., Vernon, M. L.
- Examinig Variety in Organizational Behavior Research: Psychological Differences Between Respondents...; 2004; Mestdagh, S., Buelens, M.
- Internet-Based Experiments: Prospects and Possibilities for Behavioral Accounting Research; 2004; Bryant, S. M., Hunton, J. E., Stone, D. N.
- Psychological Research Online; 2004; Kraut, R., Olson, J., Banaji, M., Bruckman, A., Cohen, J., Couper, M. P.
- Human Research And Data Collection Via The Internet; 2004; Birnbaum, M. H.
- Psychological Testing on the Internet: New Problems, Old Issues; 2004; Naglieri, J. A., Drasgow, F., Schmit, M., Handler, L., Prifitera, A., Margolis, A., Velasquez, R.
- The antecedents of consumers' loyalty toward Internet Service Providers; 2004; Chiou, J. -S.
- The impact on interviewee behavior caused by audiovisual online-communication answering sensitive questions...; 2004; Muehlenfeld, H. -U.
- Are online-offline differences in personality test scores due to increased self-disclosure?; 2004; Buchanan, T., Joinson, A. N.
- Should We Trust Web-Based Studies? A Comparative Analysis of Six Preconceptions About Internet Questionnaires...; 2004; Gosling, S. D., Vazire, S., Srivastava, S., John, O. P.
- Survey Response Behavior Shirking in Internet and Telephone Surveys; 2003; VanBeselaere, C.
- Psychologische Forschung zum und im Internet; 2003; Reips, U. -D.
- Is the Applicability of Web-Based Surveys for Academic Research limited to the Field of Information...; 2003; Roztocki, N., Lahri, N. A.
- Patterns of Drug Use and the Influence of Gender on Self-Reports of Memory Ability in Ecstasy Users:...; 2003; Rodgers, J., Buchanan, T., Scholey, A. B., Heffernan, T. M., Ling, J., Parrott, A. C.
- Internet Use and Misuse: Preliminary Findings from a New Assessment Instrument; 2003; Rotunda, R. J., Kass, S. J., Sutton, M. A., Leon, D. T.
- Show Your Real Face: A Fan Study of the UK Big Brother Transmissions (2000, 2001, 2002). Investigating...; 2003; Jones, J. M.
- Express: A Web-based technology to support human and computational experimentation; 2003; Yule, P., Cooper, R.P.
- The Use of the Internet in Psychological Research: Comparison of Online and Offline Questionnaires; 2003; Riva, R., Teruzzi, T., Anolli, L.
- Internet-based Questionnaire Assessment: Appropriate Use in Clinical Contexts; 2003; Buchanan, T.
- Digital Makeover; 2003; Sloane, J.
- Survey tools take employees' pulse instantly; 2003; Gunsauley, C.
- Internet Research: An Opportunity to Revisit Classic Ethical Problems in Behavioral Research; 2003; Pittenger, D. J.
- Presenting and Monitoring a Gender-Defined Self on the Internet; 2003; Samp, J. A., Wittenberg, E. M., Gillett, D. L.
- Best practices in employee surveying: A practitioner's perspective; 2003; Jepsen, D. M., Langford, P.
- Waiting online: a review and research agenda; 2003; Ryan, G., Valverde, M.
- A Comparison of the Psychometric Qualities of Surveys Administered by Web and Traditional Methods; 2003; Mertler, C. A., Earley, M. A.
- Validating Internet research: A test of the psychometric equivalence of Internet and in-person samples...; 2003; Meyerson, P., Tryon, W. W.
- Web-based research: Methodological variables' effects on dropout and sample characteristics; 2003; O'Neil, K. M., Penrod, S. D., Bornstein, B. H.
- Open-ended vs. Close-ended Questions in Web Questionnaires; 2003; Reja, U., Lozar Manfreda, K., Hlebec, V., Vehovar, V.
- Submit, Resist, or Subvert?: Organization Members’ Responses to Electronic Surveillance; 2003; Coopman, S., Watkins Allen, M., Hart, J.
- An empirical study on the adoption of information appliances with a focus on interactive TV; 2003; Choi, H., Choi, M., Kim, J., Yu, H.
- Humanizing self-administered surveys: experiments on social presence in web and IVR surveys; 2003; Tourangeau, R., Couper, M. P., Stegier, D.M.Steiger, D. M.; de Rouvray, C.
- Using Internet-Based Surveys With Physicians, What Works and What Doesn't Work; 2003; Schneiderman, M., Thran, S., Adams, C., Lerner, B.
- Psychological Research Online: Opportunities and Challenges; 2003; Kraut, R., Olson, J., Banaji, M., Bruckman, A., Cohen, J., Couper, M. P.
- The Impact of Lottery Incentives on Student Survey Response Rates; 2003; Whitcomb, M. E., Porter, S. R.
- Online longitudinal survey research. Validity and participation; 2002; Hiskey, S., Troop, N. A.
- E-Research: Ethics, security, design, and control in psychological research on the Internet; 2002; Banaji, M., Greenwald, A., Nosek, B.
- July 2002 Status of Forces Survey of Active-Duty Members: Tabulations of Responses; 2002
- Web-based data collection: Ethical and practical considerations; 2002; Sedivy, S. K., Brondino, M. J.
- Standards for Internet-Based Experimenting; 2002; Reips, U. -D.
- Rape Myth Acceptance and Judgments of Vulnerability to Sexual Assault: An Internet Experiment; 2002; Bohner, G., Danner, U. N., Siebler, F., Samson, G. B.
- Measuring Differences in Preactivation on the Internet: The Content Category Superiority Effect; 2002; Eichstaedt, J.
- Internet Experts' Planning of Explanations for Laypersons: A Web Experimental Approach in the Internet...; 2002; Nueckles, M., Bromme, R.
- Time Measurement And Pre-testing In On-line-Questionnaires; 2002; Mueller, J., Reimer, M.
- Using the Internet for Organizational Research: A Study of Cynicism in the Workplace; 2002; Eaton, M. A. J., Struthers, C. V.
- Internet-based Survey Research in the U.S. Navy; 2002; Olmsted, M. G.
- Attitudes of Business Faculty Towards Two Methods of Collecting Teaching Evaluations: Paper vs. Online...; 2002; Dommeyer, C. J., Baum, P., Chapman, K.S., Hanna, R.W.
- Network ethnography and the hypermedia organization: new media, new organizations, new methods; 2002; Howard, P. N.