Web Survey Bibliography
(a) Relevance & Research Question: Rating scales can considerably affect data quality regarding mean ratings, distribution of answers, response time, or item nonresponse (e.g., Couper, Conrad, & Tourangeau, 2007; Healey, 2007; Heerwegh & Loosveldt, 2002; Krosnick, 1999; Krosnick & Fabrigar, 1997). However, because implementation is easy, designers of Web surveys are tempted to use special rating scales without knowing much about their impact on data quality. This presentation focuses on how slider scales may harm survey data. Nevertheless, sometimes changes in rating scales are inevitable, especially when scrolling on Internet devices with an upright display (e.g., smart phones) should be avoided (for problems see Couper, Tourangeau, Conrad, & Crawford, 2004).
(b) Methods & Data: In a 2 x 2 Web experiment, type of rating scale (5-point Java-based slider versus 5-point HTML radio button scale) was manipulated as well as the spatial orientation on the screen. On a single Web page, respondents ( N = 779) had to evaluate two product concepts, counterbalanced for order. For analysis, respondents’ reported education was recoded in two groups, below college degree (e.g., B.A. or B.S.) and at least college degree.
(c) Results: Overall, break-off was significantly higher with slider scales in comparison to radio button scales, chi2(1, N = 779) = 12.81, p < .001, odds ratio = 6.92. Whereas respondents in the group with low education had problems with slider scales, chi2(1, N = 451) = 5.89, p = .018, odds ratio = 5.45, no difference in break-off was observed in the group of respondents with a high formal education, chi2(1, N = 321) = 1.66, p = 1.000. Additionally, task duration was considerably higher with slider scales, F (1, 703) = 638,23, p < .001, eta2 = .48. Furthermore, fewer respondents chose the middle category with slider scales. Spatial orientation of the rating scale had no significant influence on break-off or distribution of values.
(d) Added Value: The interaction between rating scale and educational level is a serious argument against the use of Java-based slider scales in general. Overall, it seems that horizontal and vertical layout can be substituted mutually.
Conference Homepage (abstract)
Web Survey Bibliography - Marketing/business (736)
- Where does the Fair Trade price premium go? Confronting consumers' request with reality; 2013; Langen, N., Adenaeuer, L.
- Customer satisfaction in Web 2.0 and information technology development; 2013; Sharma, G., Baoku, L.
- Research staff and public engagement: a UK study; 2013; Davies, S.
- Response Burden in Official Business Surveys: Measurement and Reduction Practices of National Statistical...; 2013; Giesen, D., Bavdaz, M., Loefgren, T., Raymond-Blaess, V.
- How Mobile Stacks Up to Traditional Online: A Comparison of Studies; 2013; Knowles, R.
- Book Review: Brand Together: How Co-creation Generates Innovation and Re-energizes Brands, by Nicholas...; 2013; Wilson, A.
- Digging deeper: using implicit tests to define consumers' semantic network; 2013; Riviere, P., Cuny, C., Allain, G., Vereijken, C.
- Conceptualising and evaluating experiences with brands on Facebook; 2013; Smith, S.
- Methodological Issues in the Design of Online Surveys for Measuring Unethical Work Behavior: Recommendations...; 2013; Wouters, K., Maesschalck, J., Peeters, C. F. W., Roosen, M.
- The Challenges and Benefits of Distance Mentoring; 2013; Lach, H. W. et al
- How incentives affect web-based survey response rates of athletic program donors; 2013; Alvarado, G., Callison, C.
- Permission email messages significantly increase gambler retention; 2013; Jolley, W., Lee, A., Mizerski, R., Sadeque, S.
- How virtual corporate social responsibility dialogs generate value: A framework and propositions; 2013; Korschun, D., Du, S.
- Customer loyalty to a commercial website: Descriptive meta-analysis of the empirical literature and...; 2013; Toufaily, E., Ricard, L., Perrien, J.
- Discovering interest groups for marketing in virtual communities: An integrated approach; 2013; Wang, K. -Y., Wu, H. -J., Ting, I -H.
- The Gamification of Marketing Researc; 2013; Donato, P., Link, M. W.
- Measuring Up: Impact of mobile and segmentation on respondent behaviour; 2013; Luck, K.
- Service with an e-smile: Employee authenticity and customer use of web-based support services; 2013; Turel, O., Connelly, C. E., Fisk, G. M.
- Responsive design for mixed-mode panel data; 2013; Bianchi, A., Biffignandi, S.
- Comparative analysis of data from web and face-to-face surveys. A case study on e-commerce in young...; 2013; Cappello, C., Pellegrino, D.
- Insights into Action Profiling shopping occasions for retailers through mobile and online research; 2013; Churkina, O., Morris, T.
- Mobilizing your Branded Panel: Panel data quality during the smartphone transition; 2013; Kugel, C., Brien, D., Blechman, J.
- The Attention Span of a Goldfish: Impacts of Survey Mobilization on Respondents ; 2013; Luck, K.
- Möglichkeiten zur impliziten Messung von Emotionen am Beispiel webcambasierter Gesichtsausdruckserkennung...; 2013; Wachenfeld, A., Moentmann, A., Bernet, F.
- Matching Data Collection Method to Purpose: In the Moment Data Collection with Mobile Devices for Occasioned...; 2013; Shea, C., Roberts, M., Johnson, E. P., Hadlock, W.
- The 2012 Confirmit Annual Market Research Software Survey; 2013; Macer, T., Wilson, S.
- Using Mixed-Mode Contacts in Client Surveys: Getting More Bang for Your Buck; 2013; Israel, G. D.
- ESSnet Data: Collection for Social Surveys using Multiple Modes; 2013; Sattelberger, S., Blanke, K.
- The Influence of Answer Box Format, Personal Topic Interest, and Respondent Characteristics on Response...; 2013; Keusch, F.
- Effects of Displaying Videos on Measurement in a Web Survey; 2013; Mendelson, J., Gibson, J. L., Romano Bergstrom, J. C.
- Can Google Consumer Surveys Help Pre-Test Alternative Versions of a Survey Question?: A Comparison of...; 2013; Stern, M. J., Welch, W. W.
- Tablets and Smartphones and Netbooks, Oh My! Effects of Device Type on Respondent Behavior; 2013; Ross, H., Mendelson, J., Lackey, M.
- Lotteries and study results in market research online panels; 2013; Goeritz, A., Luthe, S. C.
- Survey quality prediction system 2.0; 2013
- A nationwide web-based freight data collection; 2013; Samimi, A., Mohammadian, A., Kawamura, K.
- Measuring Wages Worldwide: Exploring the Potentials and Constraints of Volunteer Web Surveys; 2013; Steinmetz, S., Raess, D., Tijdens, K., de Pedraza, P.
- Managing mobile research: How it's different and why it matters; 2013; Kachhi-Jiwani, D., Tucker, J., Wilding-Brown, L.
- An approach to selecting online respondents; 2013; Terhanian, G.
- By the Numbers: Theory of adaptation or survival of the fittest?; 2013; Cavallaro, K.
- Measuring wages via a volunteer web survey – a cross-national analysis of item nonresponse; 2013; Steinmetz, S., Annmaria, B.
- Measuring working conditions in a volunteer web survey; 2013; de Pedraza, P., Villacampa, A.
- Prison break: Releasing offline experiments from methodological constraints by transforming them into...; 2013; Förstel, H., Manthei, K., Mohnen, A., Berger, G.
- Research Design as an Influencing Factor for Reliability in Online Market Research; 2013; Wengrzik, J., Theuner, G.
- 'Ready to complete the survey on Facebook': Web 2.0 as a research tool in business studies; 2013; Gregori, A., Baltar, F.
- Questionnaire Design: How to Plan, Structure and Write Survey Material for Effective Market Research...; 2013; Brace, I.
- Interactive and Gaming Techniques to Improve Surveys; 2012; Dean, E., Sage, A., Lai, J. W., Link, M. W., Richards, A., Vanno, L., Henning, J.
- Comparing Face-to-Face and Online Approach: Household Recruitment of Consumer Panel in China; 2012; Jin, T., Lin, Y.-C., Duan, S., Lai, J. W.
- Worldwide online research spending; 2012
- What we can learn from unintentional mobile respondents; 2012; Peterson, G.
- Unintentional mobile respondents; 2012; Peterson, G.