Web Survey Bibliography
Survey analysts are often challenged by open-ended survey comments, which interject textual data into an otherwise quantitative analysis task. In this presentation, we describe our team’s mixed-methods approach to sampling and analyzing lengthy open-ended comments for a large-scale military survey. We hope to show participants that qualitative survey data, even in large quantities, can be analyzed rigorously and painlessly. In the summer of 2010, the U.S. Department of Defense engaged Service members about the perceived impacts on the military if Don’t Ask, Don’t Tell (the law banning openly gay individuals from military service) were repealed. Westat supported this effort with numerous data collection activities, including an online survey with a sample of 400,000 Service members. Responses to the open-ended item on this 103-item instrument were critical to understanding respondents’ perceptions on repeal, but also presented an enormous challenge: Of the approximately 115,000 respondents, over 46,000 wrote comments, many of which were more than a paragraph long. How could we analyze so much data in a cost-effective and analytically rigorous way, and that met our client’s quick turn-around timeframe? In the presentation, we first describe our sample selection process, an important facet of which was classifying respondents as having “positive,” “negative,” or “mixed” attitudes towards repeal based on their responses to 21 critical survey items. We then discuss our comment analysis, including the development of a coding scheme, our use of a qualitative software package to manage the data, and, ultimately, how the key issues differed by respondent classification.
Conference Homepage (abstract)
Web Survey Bibliography - Noncoverage & sampling (1266)
- The rise of the "connected viewer"; 2012; Smith, A., Boyles, J. L.
- The effects of item saliency and question design on measurement error in a self-administered survey; 2012; Stern, M. J., Smyth, J. D., Mendez, J.
- Smartphone ownership update: September 2012; 2012; Rainie, L.
- Research company spotlight - Mobile surveys; 2012
- Participation of mobile users in traditional online studies; 2012; Jue, A.
- More dirty little secrets of online panel research.; 2012
- Mobile email opens report 2nd half 2011; 2012
- Guide to social science data preparation. Best practice throughout the data life cycle; 2012
- GMI Pinnacle; 2012
- Social media as a data collection tool: the impact of Facebook in behavioural research; 2012; Zoppos, E.
- Smartphone Apps and User Engagement: Collecting Data in the Digital Era; 2012; Link, M. W.
- Snowball Sampling in Online Social Networks; 2012; Raissi, M., Ackland, R.
- The Use of Facebook as a Locating and Contacting Tool; 2012; McCarthy, T.
- The war against unengaged online respondents; 2012; Gittelman, S. H., Trimarchi, E.
- Qualitatively Speaking: The five absolute, no-excuse must-dos for online qualitative researchers; 2012; Rossow, A.
- By the Numbers: Lessons for using online panels in B2B research; 2012; Elsner, N.
- Experience Sampling and Ecological Momentary Assessment; 2012; Stone, A.
- Can Microtargeting Improve Survey Sampling? An Assessment of Accuracy and Bias in Consumer File Marketing...; 2012; Pasek, J.
- Cognitive Evaluation of Survey Instruments: State of the Science (Art?) and Future Directions; 2012; Willis, G. B.
- Oh, Just One More Thing … Leveraging “Leave-Behinds” in Data Collection; 2012; Link, M. W.
- Can Official Records Correct Errors in Turnout Self-reports?; 2012; Berent, M., Krosnick, J. A., Lupia, A.
- Modes of Data Collection; 2012; Tourangeau, R.
- Probability vs. Non-probability Methods; 2012; Langer, G,
- Sampling for Single and Multi-Mode Surveys using Address-Based Sampling; 2012; O'Muircheartaigh, C.
- Proxy Reporting; 2012; Cobb, C. L.
- Measure the response burden in the Swedish Intrastat system; 2012; Weideskog, F.
- Boosting Web pick-up Rates by referring to Compliance Principles ; 2012; Falnes-Dalheim, E., Haraldsen, G., Sundvoll, A.
- An experimental investigation of the effects of noncontingent and contingent incentives in recruiting...; 2012; Lavrakas, P. J., Dennis, J. M., Peugh, J., Shand-Lubbers, J., Lee, E., Peugh, J., Charlebois, O., Murakami...
- Response Mode Choice and the Hard-to-Interview in the American Community Survey; 2012; Nichols, E. M., Horwitz, R., Guarino Tancreto, J.
- Recruiting in an Internet panel using respondent driven sampling; 2012; Schonlau, M.
- A Choice in Mode: A Solution for Increasing Response Rates of Hard-to-Survey Populations?; 2012; Haan, M., Ongena, Y. P.
- The Feasibility of Conducting a Web Survey Using Respondent Driven Sampling among Transgenders in the...; 2012; Kappelhof, J.
- Use of Response Propensities; 2012; Bethlehem, J., Biffignandi, S.
- Weighting Adjustment Techniques; 2012; Bethlehem, J., Biffignandi, S.
- Probabilistic survey questions and incorrect answers: Retirement income replacement rates; 2012; van Santen, P., Alessie, R., Kalwij, A.
- Combining Mail and E-Mail Contacts to Facilitate Participation in Mixed-Mode Surveys; 2012; Israel, G. D.
- Prenotification, Incentives, and Survey Modality: An Experimental Test of Methods to Increase Survey...; 2012; Tepper, J. R., Jacob, B.
- Reaching and Hearing the Invisible: Organizational Research on Invisible Stigmatized Groups via Web...; 2012; Trau, R. N. C., Haertel, C. E. J., Haertel, G. F.
- An Overlooked Approach in Survey Research: Total Survey Error; 2012; Bautista, R.
- An assessment of equivalence between Internet and paper-based surveys: evidence from collectivistic...; 2012; Fang, J., Wen, C., Prybutok, V.
- Using screen video capture software to aide and inform cognitive interviewing; 2012; Chaney, B. H., Barry, A. E., Chaney, J. D., Stellefson, M. L., Webb, M. C.
- Costs and Errors in Fixed and Mobile Phone Surveys; 2012; Vehovar, V., Slavec, A., Berzelak, N.
- Building Your Own Online Panel Via E-Mail and Other Digital Media; 2012; Toepoel, V.
- Use of Web 2.0 to Recruit Australian Gay Men to an Online HIV/AIDS Survey; 2012; Theriault, N., Bi, P., Hiller, J. E., Nor, M.
- The World's Simplest Survey Microsimulator (WSSM); 2012; Karr, A. F., Cox, L. H.
- Survey Data Collection and Integration; 2012; Davino, C., Fabbris, L.
- Challenges of assessing the quality of a prerecruited probability-based panel of internet users in...; 2012; Struminskaya, B., Kaczmirek, L.
- Impact of Fixed Choice Design on Blockmodeling Outcomes; 2012; Znidarsic, A.
- Efficiency of Different Recruitment Strategies for Web Panels; 2012; Hansen, K. M., Pedersen, R. T.
- Improving Representativeness In Online Surveys Using A Combined Internet/Postal Approach: Evidence From...; 2012; Atorough, P., Donaldson, B., Harris, A.

