Notice: the WebSM website has not been updated since the beginning of 2018.

Web Survey Bibliography

Title Using flash type questions – stroke of luck or curse for data quality?
Year 2009
Access date 20.08.2009

Online surveys are becoming more and more interactive. Respondents can use new flash question types. Card sorting, clicking on coloured and interactive buttons, ranking of items shown as pictures – the options seems to be nearly unlimited. Research institutes and panel providers expect more fun for the respondents while answering the surveys. But what’s about the results? Are there really effects on respondents and, much more important, is there any effect on the data quality of survey results? - A number of methodological questions emerge, one of them being associated with the measurement possibilities and usability. Therefore, we conduct an empirical study aim at exploring the usability and measurement quality of different flash question types.

Based on a sample provided by an online panel, a classic test design is used with a split-half sample: One half of the sample will receive questions in the commonly used way (e.g. “HTML-Buttons/ Matrix questions”). The other half of the respondents will answer a questionnaire which is programmed with flash technology. The sample will also be quoted with demographic variables like age and gender. Against this background we analyze the perception and usability of flash compared to non-flash question types. Therefore a two-step approach is used, analyzing and comparing objective measures (i.e. termination, loading time) and respondents` subjective perceptions (liking, usability experiences).

The following research questions will be verified in our study:

a) Do response rate, completing rate, answering time of questionnaire and answer-length in open-ended questions increase when using flash technology compared to commonly used scales in online surveys?

b) Are there differences between the flash sample and the “normal” users with regard to data quality?

c) Does the acceptance of flash question types differ between groups of respondents (age and gender groups)?

Based on the results of this study we give recommendations for the use of flash question types in online surveys. Furthermore, the comparison of results deriving from objective measures with respondents’ subjective experiences and evaluation will be outlined.

Access/Direct link

Conference homepage (abstract)

Year of publication2009
Bibliographic typeConferences, workshops, tutorials, presentations
Full text availabilityAvailable on request