Web Survey Bibliography
Title Using computational linguistic resources to evaluate and improve survey questions
Author Slavec, A., Vehovar, V.
Year 2014
Access date 10.12.2014
Abstract
Purpose of the study: Writing good survey questions is a complex task and several studies have shown the importance of optimally wording survey questions. Our aim is to to explore how computational linguistics resources that are available online can help survey designers to evaluate question wording.
Design/methodology/approach: A case study was conducted on a web questionnaire for international exchange students. We evaluated word frequencies in linguistic corpora and searched for synonymous wordings in the WordNET database. We made 23 different wording changes that resulted in two versions of the questionnaire, one with low-frequency and one with high-frequency wordings. To compare the two versions we conducted a split-ballot experiment on three universities.
Findings: The groups that responded to the version that was improved with computational linguistics resources; we observed decreased drop-out rates from 30% to 20%. Moreover, students found the questionnaire less difficult and less words were not understood in the improved version. On the other hand, there are no significant differences between the two versions in terms of item-nonresponse, straightlinging, response times, enjoyment of completing the questionnaire and difficulty of generating answers to
questions. However, for some indicators the results differ between different sub-populations (incoming – outgoing student, country, language).
Originality/value: Research into designing survey questionnaires has mainly focused on structural characteristics of questions such as the type, format and order of questions and response categories. On the other hand, there is substantially less research on the linguistic aspects of words used to form a question. Moreover, the originality of the study is in the use of online computational linguistics resources in pre-testing web survey questionnaires.
Research limitations/implications: The limitation of the study is that it was conducted on a student sample. Further experiment are needed on different populations and for different languages.
Practical implications: The results will help improve the web application that we are developing for evaluating survey questionnaires. We hope it will help researchers to write better survey questionnaires.
Access/Direct link
Year of publication2014
Bibliographic typeConferences, workshops, tutorials, presentations
Web survey bibliography - Vehovar, V. (46)
- Overview: Online Surveys; 2017; Vehovar, V.; Lozar Manfreda, K.
- Introduction Breakoffs, Questionnaire Breakoffs and Web Questionnaire Length: A Metastudy ; 2016; Cehovin, G.; Vehovar, V.
- Investigating respondent multitasking in web surveys using paradata; 2016; Sendelbah, A.; Vehovar, V.; Slavec, A.; Petrovcic, A.
- Identifying and correcting question-wording problems: the case of Wageindicator; 2015; Slavec, A., Vehovar, V., Tijdens, K. G.
- e-Social Science Perspective on Survey Process: Towards an Integrated Web Questionnaire Development...; 2015; Vehovar, V., Petrovcic, A., Slavec, A.
- Questionnaire length and breakoffs in web surveys: a meta study; 2014; Vehovar, V., Cehovin, G.
- WebSM Study: Overview of Features and Pricing of Software Packages in 2014; 2014; Vehovar, V., Cehovin, G.
- WebSM Study: Survey Software in 2014; 2014; Vehovar, V., Cehovin, G., Mocnik, A.
- Data quality of questions sensitive to social-desirability bias in web surveys; 2012; Lozar Manfreda, K., Zajc, N., Berzelak, N., Vehovar, V.
- WebSM Study: Survey software features overview ; 2012; Vehovar, V., Cehovin, G., Kavcic, L., Lenar, J.
- Costs and Errors in Fixed and Mobile Phone Surveys; 2012; Vehovar, V., Slavec, A., Berzelak, N.
- Web Survey Software; 2012; Berzelak, N., Vehovar, V., Slavec, A.
- Surveying general population: What types of experiments are further needed?; 2012; Vehovar, V., Berzelak, N.
- Web panels in Slovenia; 2012; Lenar, J., Vehovar, V.
- Web survey software; 2011; Slavec, A., Berzelak, N., Vehovar, V.
- Web Survey Process within the Concept of eSocial Sciences; 2011; Vehovar, V.
- Optimization of dual frame telephone survey designs; 2011; Slavec, A., Vehovar, V.
- Comments on the paper - Geetha Garib: Perceived diversity among employees; 2011; Vehovar, V.
- Nonresponse in Web Surveys ; 2011; Lozar Manfreda, K., Berzelak, N., Vehovar, V.
- Cost-Error Optimization Model for Mixed-Mode Surveys; 2009; Vehovar, V., Berzelak, N., Lozar Manfreda, K.
- Preference for Mobile Interview Surveys? Interplay of costs, errors and biases; 2009; Vehovar, V., Slavec, A.
- National readership surveys: Moving from probability face-to-face surveys to Internet panels; 2009; Vehovar, V., Slavec, A., Petric, I., Sargac, M.
- Empirical Evaluation of Web Survey Software Tools: Powerful or Friendly?; 2009; Vehovar, V., Berzelak, N., Lozar Manfreda, K., Horvat, T., Debevc, M.
- Optimising survey costs in mixed mode environment; 2009; Vehovar, V., Berzelak, N., Lozar Manfreda, K., Belak, E.
- Nonresponse Bias in Surveys; 2009; Bethlehem, J., Vehovar, V., Stoop, I., Schouten, B., Shlomo, N., Skinner, C., Montaquila, J.
- "Mobile phone surveys in mixed mode environment: Balancing costs and errors"; 2009; Vehovar, V.
- Mobile phone surveys in mixed mode environment; 2009; Vehovar, V.
- Probabilistic methods in surveys and offical statistics; 2008; Vehovar, V., Zaletel, M., Seljak, R.
- Internet surveys; 2008; Lozar Manfreda, K., Vehovar, V.
- What Can We Achieve With 5 Euros? Optimization of Survey Data Quality Using Mixed-Mode Approaches; 2008; Berzelak, N., Vehovar, V., Lozar Manfreda, K.
- Measuring ties on online forums; 2008; Ziberna, A., Vehovar, V., Jakulin, A.
- Nonresponse in web surveys within the context of survey errors and survey costs; 2008; Berzelak, N., Lozar Manfreda, K., Vehovar, V.
- Integration of the web into survey data collection: Balancing costs and errors; 2008; Vehovar, V., Berzelak, N., Lozar Manfreda, K.
- Optimization of costs and errors in mixed-mode surveys; 2008; Vehovar, V., Berzelak, N., Lozar Manfreda, K.
- Overview: Online Surveys; 2008; Vehovar, V., Lozar Manfreda, K.
- Measuring ego-centered social networks on the web: Questionnaire design issues ; 2008; Vehovar, V., Lozar Manfreda, K., Koren, G., Hlebec, V.
- The Impact of Cognitive and Computer Skills on Data Quality in Computer Assisted Self Administered Questionnaires...; 2008; Brecko, B. N., Vehovar, V.
- Internet surveys; 2008; Vehovar, V., Lozar Manfreda, K., Koren, G.
- The influence of mobile telephones on telephone surveys; 2008; Kuusela, V., Callegaro, M., Vehovar, V.
- Collecting Data on Ego-Centered Social Networks on the Web: Methodological Issues; 2004; Lozar Manfreda, K., Vehovar, V., Hlebec, V.
- Mode Effect in Web Surveys; 2002; Vehovar, V., Lozar Manfreda, K.
- Nonresponse in Web Surveys; 2002; Vehovar, V., Lozar Manfreda, K., , Batagelj, Z.
- Virtual Selves and Web Surveys; 2001; Lozar Manfreda, K., Couper, M. P., Vohar, M., Rivas, S., Vehovar, V.
- Nonresponse in Web Surveys; 1999; Vehovar, V.
- Internet v Sloveniji - Projekt RIS 1996-1998.; 1998; Vehovar, V.
- Handling "Don't Know" Survey Responses: The Case of the Slovenian Plebiscite; 1995; Rubin, D. B.; Vehovar, V.; Hal, S. S.