Notice: the WebSM website has not been updated since the beginning of 2018.

Web Survey Bibliography

Title Making Conjoint questionnaires more realistic: the effect of random noise and visual presentation on data quality and survey evaluation
Year 2017
Access date 14.04.2017

Relevance & Research Question: Conjoint is a commonly used advanced market research technique to estimate preference share for products and services with different attributes and levels. A common criticism of conjoint analysis is the repetitive nature of the questions. The main goal of this research is to study the impact on the answers and the data quality of making conjoint questions more realistic by introducing some randomised textual noise into the descriptions of the conjoint levels or by simulating the way an e-commerce website displays products.

Methods & Data: An experiment was implemented in Spain using 1,600 respondents from the opt-in online panel Netquest. The respondents were randomly assigned to one of the following four conditions: classic conjoint design without noise (control group), classic conjoint design with some random textual noise added to the attribute level descriptions, conjoint simulating e-commerce display of products but no noise, conjoint simulating e-commerce display and some random textual noise.

Results: The four groups were compared in terms of data quality and survey experience. Results show a directional (but not statistically significant) improvement of quality of estimations. In terms of survey evaluation, a Chi2 test indicates that there is a significant relationship between the experimental group and three aspects of the evaluation: finding the survey interesting (e-commerce layout without noise got the higher evaluation), liking participating in it (e-commerce layout with noise got the higher evaluation), and finding it short (e-commerce layout without noise got the higher evaluation).

Added Value: This study indicates that there is room for improvement in the design of Conjoint experiments. Even if the gains are not as high as expected, the costs of adding noise or presenting the conjoint more like an e-commerce website are low, so it seems recommendable to do it.

Year of publication2017
Bibliographic typeConferences, workshops, tutorials, presentations

Web survey bibliography (8390)