Web Survey Bibliography
Survey analysts are often challenged by open-ended survey comments, which interject textual data into an otherwise quantitative analysis task. In this presentation, we describe our team’s mixed-methods approach to sampling and analyzing lengthy open-ended comments for a large-scale military survey. We hope to show participants that qualitative survey data, even in large quantities, can be analyzed rigorously and painlessly. In the summer of 2010, the U.S. Department of Defense engaged Service members about the perceived impacts on the military if Don’t Ask, Don’t Tell (the law banning openly gay individuals from military service) were repealed. Westat supported this effort with numerous data collection activities, including an online survey with a sample of 400,000 Service members. Responses to the open-ended item on this 103-item instrument were critical to understanding respondents’ perceptions on repeal, but also presented an enormous challenge: Of the approximately 115,000 respondents, over 46,000 wrote comments, many of which were more than a paragraph long. How could we analyze so much data in a cost-effective and analytically rigorous way, and that met our client’s quick turn-around timeframe? In the presentation, we first describe our sample selection process, an important facet of which was classifying respondents as having “positive,” “negative,” or “mixed” attitudes towards repeal based on their responses to 21 critical survey items. We then discuss our comment analysis, including the development of a coding scheme, our use of a qualitative software package to manage the data, and, ultimately, how the key issues differed by respondent classification.
Conference Homepage (abstract)
Web Survey Bibliography - European survey research associaton conference 2011, ESRA, Lausanne (84)
- Proposed Indicators to Assess Interviewer Performance (IP) in CATI Surveys; 2011; Laflamme, F.
- Quantifying Open-Ended Responses: Results from an Online Advertising Tracking Survey; 2011; Jacobe, A., Brewer, L., Vakalia, F., Turner, S., Marsh, S. M.
- Quality of responses to an open-ended question on a mixed-mode survey; 2011; Gibson, J., Vakalia, F., Turner, S.
- Open-ended questions in the context of temporary work research; 2011; Siponen, K.
- How do Respondents Perceive a Questionnaire? The Contribution of Open-ended Questions; 2011; Markou, E., Garnier, B.
- The Uses of Open-Ended Questions in Quantitative Surveys; 2011; Singer, E., Couper, M. P.
- A Mixed-Methods Approach to the Analysis of Open-Ended Comments; 2011; Robins, C. S., Lee, K., Perry, S., Berkowitz, S. G., Hintze, W.
- Agree-Disagree Response Format versus Importance Judgment; 2011; Krebs, D.
- Testing a single mode vs a mixed mode design; 2011; Laaksonen, S.
- Germans' segregation preferences and immigrant group size: A factorial survey approach; 2011; Schlueter, E., Ullrich, J., Schmidt, P.
- Errors within web-based surveys: a comparison between two different tools for the analysis of tourist...; 2011; Polizzi, G., Oliveri, A. M.
- Current state of social media use for survey research; 2011; Hill, C. A., Dean, E.
- Benefits of Structured DDI Metadata across the Data Lifecycle: The STARDAT Project at the GESIS Data...; 2011; Linne, M., Brislinger, E., Zenk-Moeltgen, W.
- Microdata Information System MISSY; 2011; Bohr, J.,
- The Use of Structured Survey Instrument Metadata throughout the Data Lifecycle; 2011; Hansen, S. E.
- DDI and the Lifecycle of Longitudinal Surveys; 2011; Hoyle, L., Wackerow, J.
- Dissemination of survey (meta)data in the LISS data archive; 2011; Streefkerk, M., Elshout, S.
- Underreporting in Interleafed Questionnaires: Evidence from Two Web Surveys; 2011; Medway, R., Viera Jr., L., Turner, S., Marsh, S. M.
- The use of cognitive interviewing methods to evaluate mode effects in survey questions; 2011; Gray, M., Blake, M., Campanelli, P., Hope, S.
- Supporting the Translation Process Online in a Cross-National Survey; 2011; Amin, A., Blom, A. G., Martens, M.
- Does the direction of Likert-type scales influence response behavior in web surveys?; 2011; Keusch, F.
- Cross-country Comparisons: Effects of Scale Type and Response Style Differences; 2011; Thomas, R. K.
- Explaining more variance with visual analogue scales: A Web experiment; 2011; Funke, F.
- A Comparison of Branching Response Formats with Single Response Formats; 2011; Thomas, R. K.
- Different functioning of rating scale formats – results from psychometric and physiological experiments...; 2011; Koller, M., Salzberger, T.
- Cognitive process in answering questions: Are verbal labels in rating scales attended to?; 2011; Menold, N., Kaczmirek, L., Lenzner, T.
- Experiments on the Design of the Left-Right Self-Assessment Scale; 2011; Zuell, C., Scholz, E., Behr, D.
- Cognitive interviewing in web surveys: the use of probing questions in cross-national web surveys; 2011; Behr, D., Braun, M., Kaczmirek, L., Bandilla, W.
- The impact of monetary incentives on completion and data quality in online surveys; 2011; van Veen, F., Goeritz, A., Sattler, S.
- Is it a good idea to optimise question format for mode of data collection? Results from a mixed modes...; 2011; Nicolaas, G., Campanelli, P., Hope, S., Jaeckle, A., Lynn, P., Nandi, A.
- The role of the interviewer in producing mode effects: results from a mixed modes experiment; 2011; Hope, S., Campanelli, P., Nicolaas, G., Lynn, P., Jaeckle, A., Nandi, A.
- The role of visual and aural stimuli in producing mode effects on answers to survey questions; 2011; Lynn, P., Hope, S., Campanelli, P., Nandi, A., Nicolaas, G., Jaeckle, A.
- Causes of Mode Effects on Survey Measurement ; 2011; Lynn, P., Campanelli, P., Nicolaas, G., Hope, S., Nandi, A., Jaeckle, A.
- Separating selection from mode effects when switching from single (CATI) to mixed mode design (CATI /...; 2011; Carstensen, J., Kriwy, P., Krug, G., Lange, C.
- Testing between-mode measurement invariance under controlled selectivity conditions; 2011; Klausch, L. T.
- Evaluating Relative Mode Effects in Mixed-Mode Surveys: Three methods to disentangle selection and measurement...; 2011; Vannieuwenhuyze, J., Loosveldt, G.
- Using propensity score matching to separate mode- and selection effects; 2011; Lugtig, P. J., Lensvelt-Mulders, G. J.
- How to ask about fixed/mobile telephone usage? Impact of question wording and analysis of corresponding...; 2011; Slavec, A., Vehovar, V.
- A mixed mode pilot on consumer barometer; 2011; Taskinen, P., Simpanen, M.
- Separation of selection bias and mode effect in mixed-mode survey – Application to the face-to...; 2011; Bayart, C., Bonnel, P.
- Optimization of dual frame telephone survey designs; 2011; Slavec, A., Vehovar, V.
- A Comparison of CAPI and PAPI through a Randomized Field Experiment; 2011; De Weerdt, J.
- Flexibility of Web Surveys: Probing 'do-not-know' over the Phone and on the Web; 2011; Hox, J., de Leeuw, E. D.
- Is a Mixed-Mode Approach to Reduce Attrition in a Longitudinal Mental Health Study (NESDA) a Good Idea...; 2011; Hoogendoorn, A., Lamers, F., Hoekstra, C., Penninx, B., Smit, J. H.
- Changing research methods in Ukraine: CATI or Mixed-Mode Surveys?; 2011; Paniotto, V., Kharchenko, N.
- The effects of mixed mode designs on simple and complex analyses; 2011; Martin, P., Lynn, P.
- Mixed-mode surveys among non-western minorities in the Netherlands: does it work?; 2011; Kappelhof, J.
- Using Address-Based Sampling and Mail Contact Methods to Obtain Web Responses from the General Public...; 2011; Dillman, D. A., Messer, B. L.
- Social Climate Survey of Tobacco Control: A mixed-mode approach; 2011; Klein, J. D., McMillen, R.
- Multiple modes in cross-national surveys – a good mix?; 2011; Fitzgerald, R., Martin, P.
