In this paper, we study differences between the four data collection methods sorting, paired comparisons, conditional rankings, and triadic combinations. Effects of the judgment task on fatigue, boredom, and learning, on similarity data, and on perceptual maps derived with Multidimensional Scaling (MDS) are investigated. The data are analyzed with MAXSCAL. For each data collection method, subjects responding to the task become fatigued and bored, whereas increases in stimulus knowledge and task insight do not occur. The amount of fatigue and boredom differs between the data collection methods. Moreover, the type of data collection method used affects both the similarity data and the MDS solution.