Modern software tools for web surveys increasingly enable fast and friendly implementation of web questionnaires and overall management of the survey process. However, there exists a general tradeoff between easiness of use and powerfulness/capability. Feature-rich solutions may require a high amount of learning and/or even some programming skills, while user-friendly and intuitive applications may offer too few features. In our study, we empirically addressed these two contradicting characteristics of web survey software tools.

We analyzed the largest online list of web surveys software tools – the WebSM database (www.websm.org). The database is continuously updated and includes more than 300 web survey tools.

Initially, we analyzed all database entries according to pricing, complexity, ownership, code availability, language, location and hosting. This provides a general picture about the software that is currently available on the market.

Next, we selected a subsample of 30 software tools from typical groups of tools and studied in more details their features and user-friendliness:

• We repeated a study from 2006, which evaluated and compared the availability of basic and advanced features of web survey software tools.
• Observations and corresponding statistical analyses of the number of clicks needed to create two standardized questionnaires (one simple, one with conditions) was performed.
• The effectiveness and intuitiveness of user interfaces were systematically rated by experts.
• The evaluation was also performed by the end users, who applied certain tools and created a predefined web questionnaire.

Relations between different components of evaluation and relations between different evaluation approaches were analyzed as well. This enables an empirically evaluation of the tradeoff between the two characteristics: user-friendliness and powerfulness.

Finally, we provide some guidance in establishing criteria for selection of the software. This can help users in choosing the optimal software solutions, which fulfill their needs for functionality, usability and cost-effectiveness.