Notice: the WebSM website has not been updated since the beginning of 2018.

Web Survey Bibliography

Title A method of automated nonparametric content analysis for social science
Source American Journal of Political Science, 54, 1, pp. 229-247
Year 2010
Access date 26.02.2014
Full text

pdf (564 KB)

Abstract

The increasing availability of digitized text presents enormous opportunities for social scientists. Yet hand coding many blogs, speeches, government records, newspapers, or other sources of unstructured text is infeasible. Although computer scientists have methods for automated content analysis, most are optimized to classify individual documents, whereas social scientists instead want generalizations about the population of documents, such as the proportion in a given category. Unfortunately, even a method with a high percent of individual documents correctly classified can be hugely biased when estimating category proportions. By directly optimizing for this social science goal, we develop a method that gives approximately unbiased estimates of category proportions even when the optimal classifier performs poorly. We illustrate with diverse data sets, including the daily expressed opinions of thousands of people about the U.S. presidency. We also make available software that implements our methods and large corpora of text for further analysis.

Access/Direct link

Journal Homepage (abstract) / (full text)

Year of publication2010
Bibliographic typeJournal article
Print