Notice: the WebSM website has not been updated since the beginning of 2018.

Web Survey Bibliography

Title Improving cheater detection in web-based randomized response using client-side paradata
Year 2014
Access date 11.06.2014
Presentation

pdf (686 KB)

Abstract

Relevance & Research Question: Surveys sometimes include sensitive topics, e.g. sexual behavior or tax evasion. Respondents often hesitate to answer such sensitive items which results in high item non-response rates and a specific type of response error: a tendency to underreport socially undesirable and overreport desirable behavior. The randomized response technique (RRT) (Warner, 1965) is a well-known survey technique to reduce the problem of misreporting by protecting the privacy of the respondents. However, to obtain valid and reliable data, respondents have to understand and follow the technique´s instructions. Cheating detection models (e.g. Clark & Desharnais, 1998) try to identify the respondents which do not act according to the instructions of the design (and, hence, are cheating). Web surveys offer the opportunity to “observe” the respondents´ answering process by means of additional so-called paradata. In this study we present a new approach to detect cheaters using such client-side paradata (especially item response times).
Methods & Data: We conducted a web survey during the university´s open house (N=159) using the RRT to estimate the prevalence of deceiving in a partnership. To assess the individual item response times we implemented two comparable experimental situations; the classical RRT (including a sensitive question) and a similar RR design (without a sensitive question). Assuming that cheaters give quick answers without paying much attention to the content of the question we finally tested whether the individual item response times are significantly different in both settings.
Results: We found a small proportion of cheaters. The detected proportion of cheaters has an effect on the estimated proportion of people carrying the sensitive characteristic as a comparison with the unadjusted estimator shows.
Added Value: Previous research on cheating detection has focused only on the aggregated quantity and not on the individual “quality” of cheaters. The data quality of answers to sensitive questions is improved with such a cheating detection method based on an individual level. Here item response times (and other client-side paradata) could prospectively contribute to improve the estimation process.

Year of publication2014
Bibliographic typeConferences, workshops, tutorials, presentations
Print

Web survey bibliography - General Online Research Conference (GOR) 2014 (34)