Web Survey Bibliography
Visual analogue scales (VAS; in our study plain horizontal lines with verbal anchors at each end) have proven to be advantageous in comparison to measurement with discrete scales (like radio button scales or slider scales). They produce data on the level of an interval scale (Reips & Funke, in press) and even perform better regarding test-retest-reliability while producing fewer cases of dropout.
Our first study ( N = 195) follows up on the findings that (1) response times with VAS were significantly higher than with 5-point radio button scales, and (2) item nonresponse was much more frequent with VAS (see also Couper, Tourangeau, Conrad, & Singer, 2006). On the surface, both findings seem to indicate problems for VAS-based measurement: In general, response time is suggested as an indicator for respondent burden. But there are also opposing explanations. Higher response times may indicate deeper cognitive processing - in most cases a desired effect. Item nonresponse can also be positive, if it represents fewer cases of satisficing behavior (see Krosnick, 1991; Reips, 2000).
To be able to decide whether higher completion times and more items with missing data values are indicators for desired or undesired processes preceding a response we conduced a second Web experiment. Respondents ( N = 220) were randomly assigned either to a questionnaire with VAS or with horizontal scales with seven radio buttons. To assess respondent burden, we directly asked respondents about the perceived time needed to answer the questionnaire. Secondly, we implemented an inventory on the flow experience (Czikszentmihalyi, 1988) perceived with either scale.
Based on the results, we call the general assumption into question that respondent burden is correlated with the time needed to answer a questionnaire.
In the presentation, findings will be discussed in light of measurement error and the question answer process suggested by Sudman, Bradburn, and Schwarz (1996).
Visuelle Analogskalen (VAS; in unseren Untersuchungen einfache horizontale Linien mit verbal verankerten Enden) haben sich im Vergleich zu diskreten Messinstrumenten (wie Radiobutton- oder Sliderskalen) als vorteilhaft erwiesen. Messungen mit VAS zeichnen sich dadurch aus, dass ihre Daten das Niveau einer Intervallskala (Reips & Funke, in Druck) erreichen, sie eine größere Test-Retest-Reliabilität aufweisen und dass die Abbruchquote geringer ist.
Unsere erste Untersuchung ( N = 195) baut darauf auf, dass (1) Antwortzeiten mit VAS signifikant höher sind, als mit 5-stufigen Radiobuttonskalen und (2) Antwortverweigerung mit VAS häufiger auftritt (siehe auch Couper, Tourangeau, Conrad, & Singer, 2006). Bei oberflächlicher Betrachtung sind dies Indizien für Probleme mit VAS-gestützten Befragungen: Allgemein werden höhere Antwortzeiten als ein Indikator für größere Teilnehmerbelastung gesehen. Aber es gibt auch entgegengesetzte Erklärungsmodelle. Höhere Antwortzeiten können ein Indikator für tiefere kognitive Prozesse sein, was meist erwünscht ist. Auch Antwortverweigerung kann positiv sein, wenn dafür weniger Fälle von Satisficing (siehe Krosnick, 1991; Reips, 2000) auftreten.
Um entscheiden zu können, ob höhere Antwortzeiten und mehr Items mit fehlenden Datenwerten Indikatoren für erwünschte oder unerwünschte Prozesse sind, die einer Antwort vorausgehen, haben wir ein zweites Webexperiment durchgeführt. Allen Teilnehmern ( N = 220) wurde zufällig entweder ein Fragebogen mit VAS oder mit horizontalen siebenstufigen Radiobutton-Skalen zugewiesen. Um die Teilnehmerbelastung zu messen, ließen wir die Teilnehmer die Zeit schätzen, die sie für das Beantworten des Fragebogens benötigt hatten. Darüber hinaus implementierten wir ein Frageinventar, um bei beiden Skalen die Intensität des Flow-Erlebnisses (Czikszentmihalyi, 1998) zu messen.
Auf diesen Befunden aufbauend, stellen wir die allgemeine Annahme in Frage, dass Teilnehmerbelastung und die Zeit, die für die Beantwortung eines Fragebogens benötigt wird, miteinander korreliert sind.
Während der Präsentation werden alle Befunde im Licht des Messfehlers und des von Sudman, Bradburn und Schwarz (1996) vorgestellten Frage-Antwort-Prozesses betrachtet.
General online research (GOR) 2008 (abstract)
Web Survey Bibliography - Conference proceedings (240)
- Unintentional mobile respondents; 2012; Peterson, G.
- Sensitive topics in PC Web and mobile web surveys: Is there a difference?; 2012; Mavletova, A. M., Couper, M. P.
- Metering mobile usage. Insights from global Arbitron mobile trends panel; 2012; Verkasalo, H.
- Is „chapterisation“ a viable alternative to traditional progress indicators ?; 2012; Spicer, R., Dowling, Z.
- An experimental investigation of the effects of noncontingent and contingent incentives in recruiting...; 2012; Lavrakas, P. J., Dennis, J. M., Peugh, J., Shand-Lubbers, J., Lee, E., Peugh, J., Charlebois, O., Murakami...
- The smart(phone) way to collect survey data; 2011; Stapleton, C.
- The next CAPI evolution - completing web surveys on cell-enabled iPads; 2011; Dayton, J., Driscoll, H.
- Self-administered mobile surveys; 2011; Bosnjak, M.
- Online survey research: Findings, Best practices, and future research; 2011
- Moving forward, building an ethics community (Panel statement); 2011; Kenneally, E. et al.
- In search of a new approach to measure newspaper audiences in Canada: The journey continues; 2011; Crassweller, A., Rogers, J., Graves, F., Gauthier, E., Charlebois, O.
- Effects of response format on requalification for recontact studies; 2011; Thomas, R. K.
- A meta-analysis of experiments manipulating progress indicators in Web surveys; 2011; Callegaro, M., Villar, A., Yang, Y.
- Blend, balance, and stabilize respondent sources; 2011; Eggers, M., Drake, E.
- Mode Effect or Question Wording? Measurement Error in Mixed Mode Surveys; 2011; de Leeuw, E. D., Hox, J., Scherpenzeel, A.
- Using a fillable PDF together with SAS ® for questionnaire data; 2010; Donald, E.
- There is an app for that! A review of smartphone apps for marketing research; 2010; Michelson, M.
- The state of online research in the U.S.; 2010; Miller, J.
- Function follows form: Effects of response format on self-reported individuals and household disability...; 2010; Falcone, A. E., Thomas, R. K.
- Address-based sampling. Merits, design & implementation, and review of field statistics; 2010; Fahimi, M.
- A framework for understanding and applying ethical principles in network and security research; 2010; Kenneally, E., Bailey, M., Maughan, D.
- Restructuring and innovations on the survey “capacity of collective tourist accommodation”...; 2010; Santoro, M. T., Staffieri, S.
- An Analyze of the Zero Price Effect on Online Business Performance - An Research Based on the Mobile...; 2010; Liu, Y., Yuan, P.
- Dealing with Nonresponse in Survey Sampling: an Item Response Modeling Approach; 2010; Matei, A.
- Web survey design and usability; 2010; Karakoyun, F., Kurt, A. A.
- Response format effects on measurement of employment; 2009; Thomas, R. K., Dillman, D. A., Smyth, J. D.
- Getting data for (business) statistics: What's new? What's next?; 2009; Snijkers, G.
- Response Mode and Bias Analysis in the IRS’ Individual Taxpayer Burden Survey; 2009; Brick, J. M., Contos, G.,Masken, K.,Nord, R.
- Survey Mode Effects in Two Military Surveys; 2009; Yang, M., Falcone, A. E., Milan, L. M.
- Web based macroseismic survey: fast information exchange and elaboration of seismic intensity effects...; 2009; De Rubeis, V., Sbarra P., Sorrentino, D., Tosi, P.
- The representativeness of the LISS panel ; 2009; Knoef, M., de Vos, K.
- Sample factors that influence data quality; 2008; Gailey, R., Teal, D., Haechrel, E.
- An online panel as a platform for multi-disciplinary research; 2008; Scherpenzeel, A.
- Visual Design Effects on on Respondents Behaviour in Web-Surveys. A Design Experiment; 2008; Greinöcker, A.
- Effects of Privacy Assurances on the Online Measurement of Psychological Constructs; 2008; Witzki, A., Kramer, J.
- Effects of AJAX Technology in Online Questionnaires; 2008; Lütters, H., Westphal, D., Heublein, F.
- How Web 2.0 Technologies Can Become a Valuable Part of Online Research; 2008; Jaron, R.
- Respondent Authenticity - A biometrical approach to authenticate panelists; 2008; Wachter, B., Bender, C.
- Visual Analogue Scales Versus Categorical Scales: Respondent Burden, Cognitive Depth, and Data Quality...; 2008; Funke, F.,Reips, U. -D.
- Not Mixed-Mode but Switch-Mode; 2008; Höglinger, M., Abraham, M., Arpagaus, J.
- The Impact of Cognitive and Computer Skills on Data Quality in Computer Assisted Self Administered Questionnaires...; 2008; Brecko, B. N., Vehovar, V.
- Optimal Contact Strategy in a Mail-and-Web Mixed Mode Survey; 2008; Holmberg, A., Lorenc, B., Werner, P.
- 10 Years of Meinungsplatz.de: Success in the Collection of Data for Targeted Audiences, Such as the...; 2008; Weyergraf, O.
- Self-selection in Online Access Panels: No “Little Difference” in the Recruiting Process...; 2008; Wirth, T.
- Selectivity or Usefulness of Online-Surveys? A Comparison with Data from a Local Population Survey; 2008; Kroll, S., Hüfken, V., Reuband, K.H.
- Mobile Market Research; 2008; Maxl, E.
- Flexible and High-Speed Market Research through Mobile Data Collection & Online Reporting Tool; 2008; Haag, J., Volkmer, H.P.
- Online vs. Offline in Mobile Surveys; 2008; Neubarth, W., Maier, U.
- Gender-of-Interviewer Effects in Video-Enhanced Web Surveys. Results from a Randomized Field-Experiment...; 2008; Fuchs, M.
- The Online Use of Randomized Response Measurements; 2008; Snijders, C., Weesie, J.