Web content mining, the systematic analysis of social media like forums or blogs, plays an increasingly important role in online market research. More and more clients are interested in using this approach complementary to “traditional” online research.

A question often posed in relation to web content mining is whether the results of this method are representative for the target audience of the client’s product, or whether they mirror the orientation of opinion leaders or other biased sub-groups (e.g. very critical consumers)?

To answer this question empirically, a sentiment analysis for the dieting product ‘Weight Watchers’ was conducted in which the results of a web content mining study and a ‘traditional’ online survey were compared. Our hypothesis was that the themes and valence of the product evaluations are biased in the direction towards opinion leaders. To identify opinion leaders in the online survey, a scale for measuring this attribute was constructed.

For the web content mining a combination of quantitative and qualitative approaches was conducted. In the first step, about 10,000 posts from 14 forums were gathered and classified as “positive”, “negative” or “neutral” by using a support vector machine (SVM). Samples of positive, negative and neutral posts were then semi-automatically coded by means of a qualitative data analyses.

To compare both methods, the pre-codes of the online survey were used as reference for the coding of the posts. Moreover, new codes were built to cover new themes in the forums.

This procedure identified the quantitative as well qualitative differences between both methods.

Apart from the results of this comparison, the general importance of social media for the buying decision process is discussed on the basis of current market research results. And finally, the advantages and disadvantages of web content mining compared with online surveys are dealt with.