Web Survey Bibliography
(a) Relevance & Research Question: Rating scales can considerably affect data quality regarding mean ratings, distribution of answers, response time, or item nonresponse (e.g., Couper, Conrad, & Tourangeau, 2007; Healey, 2007; Heerwegh & Loosveldt, 2002; Krosnick, 1999; Krosnick & Fabrigar, 1997). However, because implementation is easy, designers of Web surveys are tempted to use special rating scales without knowing much about their impact on data quality. This presentation focuses on how slider scales may harm survey data. Nevertheless, sometimes changes in rating scales are inevitable, especially when scrolling on Internet devices with an upright display (e.g., smart phones) should be avoided (for problems see Couper, Tourangeau, Conrad, & Crawford, 2004).
(b) Methods & Data: In a 2 x 2 Web experiment, type of rating scale (5-point Java-based slider versus 5-point HTML radio button scale) was manipulated as well as the spatial orientation on the screen. On a single Web page, respondents ( N = 779) had to evaluate two product concepts, counterbalanced for order. For analysis, respondents’ reported education was recoded in two groups, below college degree (e.g., B.A. or B.S.) and at least college degree.
(c) Results: Overall, break-off was significantly higher with slider scales in comparison to radio button scales, chi2(1, N = 779) = 12.81, p < .001, odds ratio = 6.92. Whereas respondents in the group with low education had problems with slider scales, chi2(1, N = 451) = 5.89, p = .018, odds ratio = 5.45, no difference in break-off was observed in the group of respondents with a high formal education, chi2(1, N = 321) = 1.66, p = 1.000. Additionally, task duration was considerably higher with slider scales, F (1, 703) = 638,23, p < .001, eta2 = .48. Furthermore, fewer respondents chose the middle category with slider scales. Spatial orientation of the rating scale had no significant influence on break-off or distribution of values.
(d) Added Value: The interaction between rating scale and educational level is a serious argument against the use of Java-based slider scales in general. Overall, it seems that horizontal and vertical layout can be substituted mutually.
Conference Homepage (abstract)
Web Survey Bibliography - Marketing/business (693)
- Lotteries and study results in market research online panels; 2013; Goeritz, A., Luthe, S. C.
- Survey quality prediction system 2.0; 2013
- A nationwide web-based freight data collection; 2013; Samimi, A., Mohammadian, A., Kawamura, K.
- Measuring Wages Worldwide: Exploring the Potentials and Constraints of Volunteer Web Surveys; 2013; Steinmetz, S., Raess, D., Tijdens, K., de Pedraza, P.
- Managing mobile research: How it's different and why it matters; 2013; Kachhi-Jiwani, D., Tucker, J., Wilding-Brown, L.
- An approach to selecting online respondents; 2013; Terhanian, G.
- By the Numbers: Theory of adaptation or survival of the fittest?; 2013; Cavallaro, K.
- Measuring wages via a volunteer web survey – a cross-national analysis of item nonresponse; 2013; Steinmetz, S., Annmaria, B.
- Measuring working conditions in a volunteer web survey; 2013; de Pedraza, P., Villacampa, A.
- Prison break: Releasing offline experiments from methodological constraints by transforming them into...; 2013; Förstel, H., Manthei, K., Mohnen, A., Berger, G.
- Research Design as an Influencing Factor for Reliability in Online Market Research; 2013; Wengrzik, J., Theuner, G.
- 'Ready to complete the survey on Facebook': Web 2.0 as a research tool in business studies; 2013; Gregori, A., Baltar, F.
- Worldwide online research spending; 2012
- What we can learn from unintentional mobile respondents; 2012; Peterson, G.
- Unintentional mobile respondents; 2012; Peterson, G.
- The integration of facebook into class management: an exploratory study; 2012; Chou, P. N.
- The cross platform report. Q2 -2012 - US; 2012
- Screenwise panel: Frequently Asked Questions; 2012
- Research company spotlight - Mobile surveys; 2012
- Quality in market research. From theory to practice. 2nd Edition; 2012; Harding, D., Jackson, P.
- Online survey statistics for the mobile future. Updated with Q3 2012 data; 2012
- Not just playing around; 2012; Ewing, T.
- MRS Guidelines for online reseach; 2012
- Mobile usability; 2012; Nielsen, J., Budiu, R.
- Mobile email opens report 2nd half 2011; 2012
- ISO 20252. Market, opinion and social research-Vocabulary and service requirements, 2nd Edition; 2012
- Global market research 2012; 2012
- The war against unengaged online respondents; 2012; Gittelman, S. H., Trimarchi, E.
- Qualitatively Speaking: The five absolute, no-excuse must-dos for online qualitative researchers; 2012; Rossow, A.
- By the Numbers: Lessons for using online panels in B2B research; 2012; Elsner, N.
- Measure the response burden in the Swedish Intrastat system; 2012; Weideskog, F.
- Improving the quality of complex surveys: The case of the EU Labour Force Survey ; 2012; van der Valk, J.
- The re-engineering of the Structural Earnings survey process: Mixed - Mode data collection and new E...; 2012; Cardinaleschi, S., De Santis, S., Rocci, F., Spinelli, V.
- Developments and the impact of smart technology; 2012; Macer, T.
- Multi-Language Multi-Continent B2B Community Panel: How B2B research can effectively span the world; 2012; Morden, M., Accomando, E.
- Can Survey Gaming Techniques Cross Continents? Examining cross cultural reactions to creative questioning...; 2012; Puleston, J.
- Research Goes Mobile: Findings from initial smartphone application research; 2012; Dubreuil, C., Joubert, S.
- Research in the Mobile Mindset: Exploring the unexplored in the mobile research space; 2012; Willems, A., Veris, E., Verhaeghe, A.
- Probabilistic survey questions and incorrect answers: Retirement income replacement rates; 2012; van Santen, P., Alessie, R., Kalwij, A.
- Improving Representativeness In Online Surveys Using A Combined Internet/Postal Approach: Evidence From...; 2012; Atorough, P., Donaldson, B., Harris, A.
- Evidence on the Comparison of Telephone and Internet Surveys for Respondent Recruitment.; 2012; Potoglou, D., Kanaroglou, P. S., Robinson, N.
- Mobile Survey Participation Rates in Commercial Market Research: A Meta-Analysis; 2012; Bosnjak, M., Poggio, T., Becker, K. R., Funke, F., Wachenfeld, A., Fischer, B.
- “What a waste of time!” vs “Why not participate?” On sentiments by business...; 2012; Torres van Grinsven, V., Snijkers, G., Daas, P.
- Measuring work activities and skill requirements of occupations: Experiences from a European pilot study...; 2012; Tijdens, K., De Ruijter, E., De Ruijter, J.
- Myths and realities of respondent engagement in online surveys; 2012; Downes-Le Guin, T., Baker, R. P., Mechling, J., Ruyle, E.
- A beginner's guide to DIY research ; 2012; Cates, T.
- The Reliability and Validity of Alternative Customer Satisfaction Measurement Scales in PC Web and Mobile...; 2012; Chrzan, K., Saunders, T.
- Demonstration of an Integrated Sample Management System for a Mixed Mode (Paper/Web) Survey; 2012; Ullman, E., Peng, H., Helppie McFall, B.
- The Confirmit Annual Market Research Software Survey 2011; 2012; Macer, T.,
- How Likely?: Comparisons of Behavioral Intention Measurement Validity; 2012; Bremer, J., Thomas, R. K.