Web Survey Bibliography
Title The Effect of Motivating Elements on Response Strategies in Online Surveys
Author Boehme, R.
Year 2004
Access date 07.05.2004
Abstract The widespread use of online questionnaires in survey research raises the need for systematic research on the particularities of this new technology. This presentation outlines the theoretical and practical consequences from three combined online experiments (n = 968). As this research project was set up only with the intention to study online response behaviour, it provides answers to questions from multiple points of view. At first, bivariate analyses uncover new results and resolve contradictions in current research: The inclusion of graphic features like coloured icon scales (in comparison to numerical rating scales) and progress indicators lowers overall dropout rates. In addition, the precision of progress estimation was controlled as a 3-level factor (optimistic, pessimistic, and realistic). This showed a higher impact on compliance than the simple presence or absence of a progress indicator.
Secondly, the questionnaire design enables a verification of cognitive theories of survey response. Krosnick and Alwin (1987) distinguish between two basic response strategies, Optimizing and Satisficing. Their theory predicts an intervening influence of the individual motivation (in terms of cognitive effort) on response effects. Several reactive and non-reactive indicators (e.g. number of backups, response latency, and size of response-order effects) were evaluated with multiple logistic regression models. The results show that the offline approved model also leads to a good explanation for online response behaviour. Within this framework, a closer look at the treatments reveals that motivating elements like icon scales and progress indicators increase the overall willingness to participate but simultaneously decrease the cognitive effort spent on the response task: Researchers may pay for higher response rates with a loss in data quality.
Abstract - optional Der zunehmende Einsatz von Online-Fragebögen in der Umfrageforschung steigert das Bedürfnis nach systematischen Untersuchungen der Besonderheiten dieser neuen Technologie. Der Vortrag skizziert die theoretischen und praktischen Konsequenzen aus drei verknüpften Online-Experimenten (n = 968). Die Studie wurde eigens zur Erforschung von Online-Fragebogeneffekten angelegt und kann daher Forschungsfragen auf mehreren Betrachtungsebenen beantworten. Erstens zeigen bivariate Analysen neue Befunde und klären Widersprüche im bisherigen Forschungsstand: Der Einsatz von grafischen Elementen, wie farbigen Icon-Skalen (im Vergleich mit numerischen Rating-Skalen) und Fortschrittsanzeigen, führt insgesamt zu weniger Dropouts. Gleichzeitig wurde die Genauigkeit der Fortschrittsanzeige dreistufig variiert (optimistisch, pessimistisch und realistisch). Dieser Faktor hat offenbar einen stärkeren Einfluss auf die Teilnahmebereitschaft als allein die An- oder Abwesenheit der Fortschrittsanzeige.
Zweitens ermöglicht das Fragebogendesign die Überprüfung von kognitiven Theorien zur Beantwortung von Umfragen. Krosnick und Alwin (1978) unterscheiden zwei grundlegende Antwortstrategien, Optimizing und Satisficing. Ihre Theorie sagt einen intervenierenden Einfluss der individuellen Motivation (im Form von kognitiver Anstrengung) auf Antwort-Effekte voraus. Verschiedene reaktive und nicht-reaktive Indikatoren (z.B. Anzahl der Rückschritte, Antwort-Latenzzeiten und die Stärke von Reihenfolgeeffekten) wurden mit Hilfe einer Serie logistischer Regressionsanalysen ausgewertet. Die Ergebnisse zeigen, dass das offline bereits bewährte Modell auch gut zur Erklärung des Teilnahmeverhaltens bei Online-Befragungen geeignet ist. In diesem Rahmen deuten genauere Analysen mit den experimentellen Faktoren darauf hin, dass motivierende Elemente wie Icon-Skalen und Fortschrittsanzeigen zwar die Teilnahmebereitschaft erhöhen, jedoch gleichzeitig den kognitiven Aufwand bei der Beantwortung verringern: Forscher erkaufen sich eine höhere Rücklaufquote oft durch einen Verlust an Datenqualität.
Access/Direct link Homepage - conference (abstract)
Year of publication2004
Bibliographic typeConferences, workshops, tutorials, presentations
Web Survey Bibliography - 2004 (512)
- Presuming pervasive internet access: The survey research paradigm in transition; 2004; Coates, D.
- The impact on interviewee behavior caused by audiovisual online-communication answering sensitive questions...; 2004; Muehlenfeld, H. -U.
- HISBUS Online-Panel: A virtual student village as an instrument for providing political advice and flash...; 2004; Muessig Trapp, P.
- Comparison of laboratory and online experiments: A research program; 2004; Ollesch, H., Heineken, E., Schulte, F. P.
- Paper and pencil or online? Methodological Experiences from an employee survey; 2004; Poetschke, M.
- Employee surveys via Internet or paper? The influence of administration mode, anonymity, voluntariness...; 2004; Reips, U. -D., Franek, L.
- Using web surveys in mixed-mode approaches: an experimental comparison with traditional survey modes; 2004; Riek, S., Rietz, C., Kruger, T.
- Teaching online methods in higher education - the study module "Experimental Psychology" as an example...; 2004; Schulte, F. P., Heineken, E., Ollesch, H.
- "What the h... are they doing?" - What are respondents doing while filling in an online-questionnaire...; 2004; Stieger, S.
- An Overview of Capabilities and Methodological Research Conducted by the Government and Academic Area...; 2004; Dennis, J. M.
- Probability Samples vs. Volunteer Respondents in Internet Research: Defining Potential Effects on Data...; 2004; Pineau, V., Slotwiner, D.
- R U There? Using Text Messaging as a method of contact in Wireless; 2004; Buskirk, T. D., Steeh, C. G.
- Qualitative Discussion Groups: An Online Contribution to Research Methods; 2004; Dammer, I.
- Motivations underlying the intention to participate in Internet-based research; 2004; Deutskens, E., Wetzels, M., de Ruyter, K.
- Online Experiments in Commercial Market Research; 2004; Nordmeyer, C. -F., Geissler, H., Donath, T.
- Online- and offline-employee surveys in theory and practice; 2004; Escher, C., Hauser, F.
- Do access panels really yield representative results?; 2004; Faas, T.
- Mode test of an online and paper employee satisfaction survey: Comparison of respondents and non-respondents...; 2004; Gesell, S. B., Burkholz, S. D., Standiford, M. J.
- Online-Surveys: Effects of different display formats, response orders as well as progress indicators...; 2004; Glauer, R., Schneider, D.
- What They See Is What We Get: Response Options for Web Surveys; 2004; Tourangeau, R., Crawford, S. D., Conrad, F. G., Couper, M. P.
- Stereotypic Response Patterns within Matrix Questions in Web Surveys; 2004; Gockenbach, S., Bosnjak, M., Goeritz, A.
- Material Incentives in Web Surveys: Two Meta-Analyses; 2004; Goeritz, A.
- Salutation, Power and behaviour in on-line panels; 2004; Joinson, A. N., Reips, U. -D.
- Web-based surveys in market and social research - usage and needs of different user groups in the EU; 2004; Kaczmirek, L., Bosnjak, M., Bandilla, W., Auer, T.
- Conceptual brand m@pping - A web-based method to elicit conceptual networks of brand knowledge and to...; 2004; Klein-Reesink, T.
- Website optimisation & controlling by benchmarking; 2004; Knapp, F.
- Application of image-based conjoint analysis on the internet; 2004; Koch, T.
- E-voting: participation, turn out, and digital divide; 2004; Oostveen, A.-M., Besselaar, P.
- The Effect of Motivating Elements on Response Strategies in Online Surveys; 2004; Boehme, R.
- Online and other methods of data collection in employee surveys: A comparison; 2004; Borg, I., Faulbaum, F.
- Are online-offline differences in personality test scores due to increased self-disclosure?; 2004; Buchanan, T., Joinson, A. N.
- Telephone versus Internet samples for a national advisory referendum: are the underlying stated preferences...; 2004; Li, H., Berrens, R. P., Bohara, A. K., Silva, C. L., Weimer, D. L., Jenkins-Smith, H. C.
- Should We Trust Web-Based Studies? A Comparative Analysis of Six Preconceptions About Internet Questionnaires...; 2004; Gosling, S. D., Vazire, S., Srivastava, S., John, O. P.
- On the integration of the Internet into science communication; 2004; Barjak, F.
- Determinants of participitation in online access panels; 2004; Batinic, B.
- Spotting online influentials among business audiences. Using online research in viral marketing campaigns...; 2004; Cakim, I.
- Publishing survey reports with XML; 2004; MacKay, I.
- Fully automated quality monitoring; 2004; Lundgren, P., Nergard, A.
- Are the mobile phone users ready for MCASI - mobile computer assisted self interviewing?; 2004; Tjostheim, I., Thalberg, S.
- What you see is what you get: An international perspective in online survey design; 2004; Thomas, R. K.
- Website design is about understanding the user: How a modal theory of user experience can help develop...; 2004; Comley, P., Lang, J.
- “The full picture” – using eye tracking technology to make web site design more effective...; 2004; Barber, H., Janes, I.
- From e-surveys to e-interviews: how to use technology to interact more intelligently with respondents...; 2004; Loewe, G.
- Sampling: the next must-have for online market researchers; 2004; Luth, R.
- Online panels are the future!; 2004; Noyce, D.
- Internet-Usage within a Delphi-study; 2004; Balzer, L.
- User and Non-user Behaviour - Illustrated by the largest Internet Retailer; 2004; Ergenzinger, R., Bamert, T.
- The impact of web page text-background colour combinations on readability, retention, aesthetics and...; 2004; Hall R. H., Hanna, P.
- An Examination of the Equivalence of Web-Based Versus Paper-and-Pencil Upward Feedback Ratings: Rater...; 2004; Smither, J.W., Walker, A.G., Yap, M.K.T.
- The role of metadata in the Statistical Knowledge Network - An emerging research agenda; 2004; Denn, S., Hs, S.W., Hert, C. A.