Web Survey Bibliography
There is concern that professional respondents, motivated more by monetary incentives than the opportunity to provide thoughtful answers, may compromise data quality in on-line surveys. The worry is that such respondents primarily take actions on the “critical path” toward their monetary goal even though valid answers may require additional, “off-path” actions. Our main questions are whether professional respondents are quicker to find the critical path and adhere to it more closely than "non-professionals." We explored this in a web survey in which 2404 volunteer respondents answered two constant-sum questions, i.e., multi-part items whose answers must add to 100% (fourteen internet use categories) or 24 hours (eleven daily activities). Respondents could not advance unless their answers were well-formed (i.e., equal to the target sum). About one third of the respondents could click for a running tally to facilitate well-formedness, placing this action on the critical path. We considered respondents who belonged to more than 3 volunteer panels to be professional (58% had completed more than 30 web surveys). This distinction mattered. Professionals were more likely than non-professionals to request a tally for the first item but the quality of both groups’ responses benefited from the tally: more non-zero, non-missing answers and fewer rounded responses with than without the tally. However, by the second item, professional respondents seemed to learn that providing valid answers was not on the critical path. While professionals again requested the tally more often, the tally only improved the quality of answers for nonprofessionals: more non-zero, non-missing answers and less time spent on “other” activities with than without the tally; professionals’ answers showed no such improvement from the tally. The results suggest that professional respondents are more likely to provide well-formed than valid answers because validity is not on the critical path.
Conference Homepage (abstract)
Web survey bibliography - Zhang, C. (16)
- Reducing speeding in web surveys by providing immediate feedback; 2017; Conrad, F.; Tourangeau, R.; Couper, M. P.; Zhang, C.
- Respondent mode choice in a smartphone survey ; 2017; Conrad, F. G., Schober, M. F., Antoun, C., Yan, H. Y., Hupp, A., Johnston, M., Ehlen, P., Vickers, L...
- Helping respondents provide good answers in Web surveys; 2016; Couper, M. P.; Zhang, C.
- Comparisons of Online Recruitment Strategies for Convenience Samples: Craigslist, Google AdWords, Facebook...; 2016; Antoun, C., Zhang, C., Conrad, F. G., Schober, M. F.
- A Review of Issues in Gamified Surveys; 2015; Keusch, F.; Zhang, Che.
- Speeding in Web Surveys: The tendency to answer very fast and its association with straightlining; 2013; Conrad, F. G.; Zhang, Che.
- Satisficing in Web Surveys: Implications for Data Quality and Strategies for Reduction; 2013; Zhang, Che.
- Speeding and Non-Differentiation in Web Surveys: Evidence of Correlation and Strategies for Reduction...; 2013; Zhang, Che.
- Evaluating Interactive Feedback in Computer-Assisted Self-Interviewing (CASI); 2013; Hudson, M. L., Hupp, A., Zhang, C., Schroeder, H. M.
- Database Lookup in Web Surveys; 2012; Couper, M. P., Zhang, C., Conrad, F. G., Tourangeau, R.
- Designing Interactive Interventions in Web Surveys: Humanness, Social Presence and Data Quality; 2012; Zhang, Che.
- Impact on Data Quality of Making Incentives Salient in Web Survey Invitations.; 2011; Zhang, Che.
- Professional Web Respondents and Data Quality; 2010; Conrad, F. G., Tourangeau, R., Couper, M. P., Zhang, C.
- Differences Between Internet and Non-Internet Households on Survey Items: Do These Differences Disappear...; 2009; Zhang, C., Callegaro, M., Thomas, M.
- Do we hear different voices?: Investigating the differences between internet and non-internet users...; 2009; Zhang, C., Callegaro, M., Thomas, M., DiSogra, C.
- More than the digital divide?: Investigating the differences between Internet and non-Internet users; 2008; Zhang, C., Callegaro, M., Thomas, M.