Web Survey Bibliography
(a) Relevance & Research Question: Rating scales can considerably affect data quality regarding mean ratings, distribution of answers, response time, or item nonresponse (e.g., Couper, Conrad, & Tourangeau, 2007; Healey, 2007; Heerwegh & Loosveldt, 2002; Krosnick, 1999; Krosnick & Fabrigar, 1997). However, because implementation is easy, designers of Web surveys are tempted to use special rating scales without knowing much about their impact on data quality. This presentation focuses on how slider scales may harm survey data. Nevertheless, sometimes changes in rating scales are inevitable, especially when scrolling on Internet devices with an upright display (e.g., smart phones) should be avoided (for problems see Couper, Tourangeau, Conrad, & Crawford, 2004).
(b) Methods & Data: In a 2 x 2 Web experiment, type of rating scale (5-point Java-based slider versus 5-point HTML radio button scale) was manipulated as well as the spatial orientation on the screen. On a single Web page, respondents ( N = 779) had to evaluate two product concepts, counterbalanced for order. For analysis, respondents’ reported education was recoded in two groups, below college degree (e.g., B.A. or B.S.) and at least college degree.
(c) Results: Overall, break-off was significantly higher with slider scales in comparison to radio button scales, chi2(1, N = 779) = 12.81, p < .001, odds ratio = 6.92. Whereas respondents in the group with low education had problems with slider scales, chi2(1, N = 451) = 5.89, p = .018, odds ratio = 5.45, no difference in break-off was observed in the group of respondents with a high formal education, chi2(1, N = 321) = 1.66, p = 1.000. Additionally, task duration was considerably higher with slider scales, F (1, 703) = 638,23, p < .001, eta2 = .48. Furthermore, fewer respondents chose the middle category with slider scales. Spatial orientation of the rating scale had no significant influence on break-off or distribution of values.
(d) Added Value: The interaction between rating scale and educational level is a serious argument against the use of Java-based slider scales in general. Overall, it seems that horizontal and vertical layout can be substituted mutually.
Conference Homepage (abstract)
Web Survey Bibliography - 2011 (584)
- Involve while you evolve. How to make mobile research work for everyone; 2011; Luck, K.
- Internet access quarterly update 2011 Q1; 2011
- In search of a new approach to measure newspaper audiences in Canada: The journey continues; 2011; Crassweller, A., Rogers, J., Graves, F., Gauthier, E., Charlebois, O.
- Households with Computers, Telephone Subscriptions, and Internet Access, Selected Years, 1997 - 2010; 2011
- Harvard research data security policy; 2011
- Handbook of statistical data editing and imputation; 2011; Waal, T., Pannekoek, J.; Scholtus, S.
- Handbook of parametric and nonparametric statistical procedures, 5th Edition; 2011; Sheskin, D. J.
- GRE® program announces big benefits and big savings for GRE® test takers worldwide; 2011
- Google and Kantar develop measurement panel; 2011
- Going online with assessment; 2011; Burke, E. et al.
- Global market research 2011; 2011
- Focus Group Methodology: Principle and Practice; 2011; Liamputtong, P.
- External preference mapping of commercial antiaging creams based on consumers' responses to a check...; 2011; Parente, M. E., Manzoni, A. V., Ares, G.
- Exploring the digital nation. Computer and Internet use at home; 2011
- Experiments for evaluating survey questions; 2011; Krosnick, J. A.
- Evaluating the usability of personal digital assistants to collect behavioral data on adolescents with...; 2011; McClamroch, K. J
- Eurobarometer Special surveys: EB75.1 E-Communications Household Survey. Special Eurobarometer 362; 2011
- Survey Data Quality Provisions in Statistics Canada E-Questionnaire Solution: Retrospective and Perspectives...; 2011; Abiza, Y.
- Developing Electronic Questionnaire Guidelines: Issues and Challenges in a Changing Environment; 2011; Cote, A.-M., Kelly, P., Lawrence, D.
- Triton: a general tool for data collection and micro editing; 2011; Erikson, J.
- A Generalized System for Aided Development and Monitoring of Web Surveys; 2011; Torelli, R.
- Ethical issues in Internet research; 2011; Hoerger, M., Currell, C.
- Using survey data collection as a tool for improving the survey process; 2011; Biffignandi, S., Perani, G., Laureti, A.
- Essential methods for design based sample surveys; 2011; Pfeffermann, D., Rao, C. R.
- ESOMAR AND CASRO submission to the W3C tracking protection working group - Market research techniques...; 2011
- A picnic in the field; negotiating the presentation of the self in researcher/respondent relationships...; 2011; Parsons, J.
- The benefits and constraints of e-mail interviews and discussions as methods of accessing valid data; 2011; Roberts, A.
- Is There a Quick Fix for Open-ended Questions? A Comparison of Qualitative Analysis Techniques; 2011; Tesfaye, C.
- The Impact of Open-Ended Questions: A Multivariate Study of Respondent Engagement; 2011; Gittelman, S. H.
- “You are Invited to Participate”: Challenges of Applying Mixed Survey Methods to Assess...; 2011; Chew, F.
- Literacy and Data Quality in Self-Administered Surveys; 2011; Smyth, J. D., Olson, K.
- Catch Them When You Can: Speeders and Their Role in Online Data Quality; 2011; Gutierrez, C., Wells, T., Rao, K., Kurzynski, D.
- Effects of Post-Incentives on Response Rates, Costs, and Response Quality in a Web Survey of College...; 2011; Stevenson, J., Dykema, J., Cyffka, K., Klein, L., Goldrick-Rab, S.
- Observed Differences in the Placement and Wording of Neutral Response Options in Web Surveys: An Experiment...; 2011; Walton, L., Cobb, C. L., DiSogra, C.
- Effects of response format on requalification for recontact studies; 2011; Thomas, R. K.
- A meta-analysis of experiments manipulating progress indicators in Web surveys; 2011; Callegaro, M., Villar, A., Yang, Y.
- Does mentioning "some people" and "other people" in a survey question increase the...; 2011; Yeager, D. S., Krosnick, J. A.
- Do not track gathers momentum; 2011; Stark, D.
- “Don’t know” the difference - An experimental comparison between Web and CATI; 2011; Schielicke, A.-M., Degen, M.
- Display resolution; 2011
- A Survey Stopping Rule Based on Weighting for Unit Nonresponse; 2011; Lewis, T.
- Classic Inspirations for Social Research Methodology in the time of Online Access Panels ; 2011; Jeřábek, H.
- Five Things You Should Know About Mobile Data Collection; 2011; Pingitore, G.
- Mixed Methods - Analyzing Open-Ended Comments in a Quantitative Employee Survey; 2011; Lawton, L., Broege, N.
- Changing Survey Methods (Discussion); 2011; Lavrakas, P. J.
- Code of standards and ethics for survey research; 2011
- Causes of survey incompletes: Why panelists say they abandon surveys; 2011; Henning, J.
- Canadian online panels: Similar or different?; 2011; Chan, P., Ambrose, D.
- Blend, balance, and stabilize respondent sources; 2011; Eggers, M., Drake, E.
- Beyond data stability: Rising above quality concerns; 2011