Web Survey Bibliography
In surveys it is problematic to get valid responses when studying topics that respondents find threatening to answer questions about (Lee, 1993). Researchers analyzing such sensitive topics are more likely to be confronted with higher non-response rates and/or socially desirable answers, which threatens the validity of results.
One of the solutions to this problem is to use a Randomized-Response (RR) technique (Warner, 1965; Kuk, 1990; Chaudhuri & Mukerjee, 1988; Fox, 1986). In this technique the answer of the respondent depends in part on a randomization device (typically the throw of two dice) and because of this the interviewer/researcher can no longer determine what the response of the individual respondent was. Because the properties of the randomization procedure are known, valid inferences about the behavior under study can still be made.
Although RR-techniques are well-known and meta-analyses show that they improve the validity of the results (Edgell et al., 1982; Lensvelt-Mulders et al, 2005), they are not often used. Several reasons are given for this in the literature. First, to get a comparable precision, RR-techniques require larger sample sizes. Second, it seems that researchers erroneously think that RR-techniques do not allow for analyses at the individual level.
The usage of RR-techniques online is in its infancy. On the one hand the anonymity of the internet is expected to increase the willingness to answer sensitive questions, on the other hand the implementation of RR-techniques online is complicated by the fact that online implementations of a randomization device are less likely to be trusted to guarantee anonymity by the respondent.
Design In our study we collected survey-data of about 3,250 respondents (initial wave of 1,000 and second wave of 750 from wave 1 and 2,250 new respondents) of a large Dutch Internet-panel (200,000+ members) who answered questions about three sets of potentially sensitive topics: traffic violations, possession of illegal software and music, and answering behavior in the online panel. Our design allows, among others, for comparisons with respect to the following issues:
- Voluntary versus involuntary randomization: For those respondents who do not mind answering the questions honestly, using the randomization procedure produces unnecessary noise. We compare direct question and standard randomization response techniques with voluntary choice as to the method of elicitation.
- Browser based versus locally executable randomization. In most online applications, RR-techniques simulate the throw of dice in the browser. Obviously, it would be easy for the researcher to monitor or manipulate the throw of the electronic dice, undermining the faith of the respondent in the procedure. We compare direct question and standard randomization response techniques with the case where the randomization device is a downloadable application.
- Determinants of compliance and test-retest reliability. Despite the procedure, respondents sometimes answer in socially desirable ways just to make sure that they do not confess to any kind of sensitive behavior. Using multiple RR-measurements, we identify which kinds of respondents are more likely to exhibit this kind of behavior. Moreover, in an initial wave we asked the same sensitive questions directly to 1,000 respondents, which allows for another way of determining compliance and test-retest reliability of the measurement procedures.
The data are being collected end of September, so we cannot present results at the time of submission of this abstract. The analysis of RR-data has thusfar been restricted to somewhat non-standard software (such as R and GLIM). Besides the issues mentioned above, in our presentation we will briefly elaborate on ways to use standard software to analyze RR-data.
General online research (GOR) 2008 (abstract)
Web Survey Bibliography - General Online Research Conference (GOR) 2008 (20)
- Visual Design Effects on on Respondents Behaviour in Web-Surveys. A Design Experiment; 2008; Greinöcker, A.
- Effects of Privacy Assurances on the Online Measurement of Psychological Constructs; 2008; Witzki, A., Kramer, J.
- Effects of AJAX Technology in Online Questionnaires; 2008; Lütters, H., Westphal, D., Heublein, F.
- How Web 2.0 Technologies Can Become a Valuable Part of Online Research; 2008; Jaron, R.
- Respondent Authenticity - A biometrical approach to authenticate panelists; 2008; Wachter, B., Bender, C.
- Visual Analogue Scales Versus Categorical Scales: Respondent Burden, Cognitive Depth, and Data Quality...; 2008; Funke, F.,Reips, U. -D.
- Not Mixed-Mode but Switch-Mode; 2008; Höglinger, M., Abraham, M., Arpagaus, J.
- The Impact of Cognitive and Computer Skills on Data Quality in Computer Assisted Self Administered Questionnaires...; 2008; Brecko, B. N., Vehovar, V.
- Optimal Contact Strategy in a Mail-and-Web Mixed Mode Survey; 2008; Holmberg, A., Lorenc, B., Werner, P.
- 10 Years of Meinungsplatz.de: Success in the Collection of Data for Targeted Audiences, Such as the...; 2008; Weyergraf, O.
- Self-selection in Online Access Panels: No “Little Difference” in the Recruiting Process...; 2008; Wirth, T.
- Selectivity or Usefulness of Online-Surveys? A Comparison with Data from a Local Population Survey; 2008; Kroll, S., Hüfken, V., Reuband, K.H.
- Mobile Market Research; 2008; Maxl, E.
- Flexible and High-Speed Market Research through Mobile Data Collection & Online Reporting Tool; 2008; Haag, J., Volkmer, H.P.
- Online vs. Offline in Mobile Surveys; 2008; Neubarth, W., Maier, U.
- Gender-of-Interviewer Effects in Video-Enhanced Web Surveys. Results from a Randomized Field-Experiment...; 2008; Fuchs, M.
- The Online Use of Randomized Response Measurements; 2008; Snijders, C., Weesie, J.
- The Influence of Human Cues on Drop-out and Answer Behaviour within Web Surveys; 2008; Oesterlau, T., Geißler, H.
- “For Example…,” How Different Example Types in Online Surveys Influence Frequency...; 2008; Berent, M., Krosnick, J. A.
- What They see is what You get: Nuts and Bolts of Web surveys; 2008; Baker, R. P.