Relevance & Research Question: To date, there exist only a small number of articles which have set out to review and evaluate existing online survey software packages and their features (e.g. Gordon, 2002; Wright, 2005). Furthermore, many of the available reviews do not engage in detailed testing of specific packages within a social/behavioural research framework. Few reviews actually engage in any depth with the extant literature outlining guidelines and principles of good practice for online survey research (for exceptions see Crawford, 2002; Kaczmirek, 2008). The present research adds to and extends these existing review/evaluation studies by carrying out an in-depth testing and evaluation of three popular, low cost, packages, within a context of good design practice and principles in social and behavioural online research.
Methods & Data: First, a large number of online survey software packages were sourced (using resources such as: http://www.websm.org/), and their features assessed against an initial screening list of 'essential criteria', derived with reference to literature on guidelines for good design practice in online survey/research methods. Emphasis was placed on features which support procedures that help maximise data integrity and adherence to ethical guidelines, as well as evidence of longevity, affordability and ongoing development. Second, three packages were selected and each used to implement a) a simple survey design, and b) a simple experimental design. Extensive testing was carried out in order to provide a detailed list of features, functionalities, strengths and weaknesses of each package.
Results: It was concluded that each package reviewed had it's own strengths and weaknesses, research design contexts in which it may be more or less useful, and caveats concerning where it may prove most problematic. Conclusions on which of the packages would prove most suitable for different researcher's needs in different design contexts are offered.
Added Value: The present research fills a gap in the literature by presenting the first (to the best of the author's knowledge) attempt to situate evaluation of online survey software packages firmly within a context of social/behavioural research, by using three such packages to implement actual research study designs.