Web Survey Bibliography
Survey analysts are often challenged by open-ended survey comments, which interject textual data into an otherwise quantitative analysis task. In this presentation, we describe our team’s mixed-methods approach to sampling and analyzing lengthy open-ended comments for a large-scale military survey. We hope to show participants that qualitative survey data, even in large quantities, can be analyzed rigorously and painlessly. In the summer of 2010, the U.S. Department of Defense engaged Service members about the perceived impacts on the military if Don’t Ask, Don’t Tell (the law banning openly gay individuals from military service) were repealed. Westat supported this effort with numerous data collection activities, including an online survey with a sample of 400,000 Service members. Responses to the open-ended item on this 103-item instrument were critical to understanding respondents’ perceptions on repeal, but also presented an enormous challenge: Of the approximately 115,000 respondents, over 46,000 wrote comments, many of which were more than a paragraph long. How could we analyze so much data in a cost-effective and analytically rigorous way, and that met our client’s quick turn-around timeframe? In the presentation, we first describe our sample selection process, an important facet of which was classifying respondents as having “positive,” “negative,” or “mixed” attitudes towards repeal based on their responses to 21 critical survey items. We then discuss our comment analysis, including the development of a coding scheme, our use of a qualitative software package to manage the data, and, ultimately, how the key issues differed by respondent classification.
Conference Homepage (abstract)
Web Survey Bibliography - Costs (461)
- Methodological Considerations of Qualitative Email Interviews; 2013; Nehls, K.
- Compared to a small, supervised lab experiment, a large, unsupervised web-based experiment on a previously...; 2013; Ryan, R. S., Wilde, M., Crist, S.
- Managing mobile research: How it's different and why it matters; 2013; Kachhi-Jiwani, D., Tucker, J., Wilding-Brown, L.
- A standard for test reliability in group research; 2013; Ellis, J. L.
- The General Survey System Initiative at RTI International: An Integrated System for the Collection and...; 2013; Thalji, L., Mitchell, S., Hill, C. A., Suresh, R., Speizer, H., Pratt, D.
- Innovation in Data Collection: the Responsive Design Approach; 2013; Bianchi, A., Biffignandi, S.
- Measurement effects in mixed-mode panel surveys; 2013; Lugtig, P. J.
- The ONS Beyond 2011 Programme & possible implications for social surveys; 2013; Morris, L.
- Issues of Coverage and Sampling in Web Surveys for the General Population: An Overview; 2013; Lynn, P.
- Use of a Social Networking Web Site for Recruiting Canadian Youth for Medical Research; 2013; Chu, J. L., Snider, C. E.
- Measurement invariance and quality of composite scores in a face-to-face and a web survey; 2013; Revilla, M.
- Convenient yet not a convenience sample: Jury pools as experimental subject pools; 2013; Murray, G. R., Rugeley, C. R., Mitchell, D.-G., Mondak, J. J.
- Online Survey Software; 2013; Baker, J. D.
- Incentives for college student participation in web-based substance use surveys; 2013; Patrick, M. E., Singer, E., Boyd, C. J., Cranford, J. A., McCabe, S. E.
- Screenwise panel: Frequently Asked Questions; 2012
- Social media as a data collection tool: the impact of Facebook in behavioural research; 2012; Zoppos, E.
- Social research in online context: methodological reflections on web surveys from a case study; 2012; Pandolfini, V.
- Can Microtargeting Improve Survey Sampling? An Assessment of Accuracy and Bias in Consumer File Marketing...; 2012; Pasek, J.
- Optimizing Response Rates; 2012; Brick, J. M.
- Modes of Data Collection; 2012; Tourangeau, R.
- The re-engineering of the Structural Earnings survey process: Mixed - Mode data collection and new E...; 2012; Cardinaleschi, S., De Santis, S., Rocci, F., Spinelli, V.
- Between demand and reality: Ensuring efficiency and quality in pretesting questionnaires; 2012; Sattelberger, S., Blanke, K.
- Choosing a Data Collection Approach: Mixed Mode Design Experiences in Statistics Finland; 2012; Taskinen, P., Kiianmaa, N.
- An experimental investigation of the effects of noncontingent and contingent incentives in recruiting...; 2012; Lavrakas, P. J., Dennis, J. M., Peugh, J., Shand-Lubbers, J., Lee, E., Peugh, J., Charlebois, O., Murakami...
- The Feasibility of Conducting a Web Survey Using Respondent Driven Sampling among Transgenders in the...; 2012; Kappelhof, J.
- WEBDATANET: web-based data-collection methodological challenges, solutions and implementations. Action...; 2012; de Pedraza, P.
- WebSM Study: Survey software features overview ; 2012; Vehovar, V.; Cehovin, G.; Kavcic, L.; Lenar, J.
- Web Panels; 2012; Bethlehem, J., Biffignandi, S.
- Weighting Adjustment Techniques; 2012; Bethlehem, J., Biffignandi, S.
- Mixed-Mode Surveys; 2012; Bethlehem, J., Biffignandi, S.
- Effects of E-Mailed Versus Mailed Invitations and Incentives on Response Rates, Data Quality, and Costs...; 2012; Dykema, J., Stevenson, J., Klein, L., Kim, Y., Day, B.
- Prenotification, Incentives, and Survey Modality: An Experimental Test of Methods to Increase Survey...; 2012; Tepper, J. R., Jacob, B.
- Costs and Errors in Fixed and Mobile Phone Surveys; 2012; Vehovar, V., Slavec, A., Berzelak, N.
- Paper-and-Pencil versus Web Administration of a Student Satisfaction Survey; 2012; Bowen, C.-C.
- An Initial Look at Non-Response and Attrition in Understanding Society; 2012; Lynn, P., Burton, J., Kaminska, O., Knies, G., Nandi, A.
- Evidence on the Comparison of Telephone and Internet Surveys for Respondent Recruitment.; 2012; Potoglou, D., Kanaroglou, P. S., Robinson, N.
- Comparison of response rates and cost-effectiveness for a community-based survey: postal, internet and...; 2012; Sinclair, M., O'Toole, J., Malawaraarachchi, M., Leder, K.
- Online Questionnaire Data Analysis System (OQDAS); 2012; Ali, A. Q.
- Solving the Mode Mystery The Cost, Coverage and Quality Tradeoffs of Picking (and Mixing) Online and...; 2012; Cape, P. J., Phillips, K.
- Modular Survey Design for Mobile Devices; 2012; Johnson, A., Kelly, F.
- Improving RDD Cell Phone Samples. Evaluation of Different Pre-call Validation Methods; 2012; Kunz, T., Fuchs, M.
- Influencing Mode Choice in a Mixed Mode Survey; 2012; Mooney, G., Lan, F., Lin, X., Hurwitz, A.
- Best Approaches to Mode Order and Non-response Prompting in a Multi-Mode Survey; 2012; Newsome, J., Levin, K., Schafer, B., Vigil, M., Liu, W.
- Better (Quality), Faster, Cheaper? Completing Web Surveys on Cell-Enabled iPads; 2012; Driscoll, H., Dayton, J. J., Pels, R.
- The Challenge of Going National: An Experimental Evaluation of the Effects of Local vs. Distant Survey...; 2012; Edwards, M. L., Dillman, D. A.
- What are the Odds? Lotteries versus Cash Incentives. Response Rates, Cost and Data Quality for a Web...; 2012; Stevenson, J., Dykema, J., Klein, L., Cyffka, K., Goldrick-Rab, S.
- Data Quality from Low Cost Data Collection Methodologies; 2012; Traugott, M. W.
- Using the iPad2 as a Prize-based Incentive to Boost Response Rates; 2012; McClendon, R., Jenson, E., Olsen, D.
- What Would You Do? Conducting Web-Based Factorial Vignette Surveys; 2012; Aviram, H.
- The “MediaLiveTracker” – A New Online-Tool for Real-Time-Response-Measurement; 2012; Kercher, J., Bachl, M., Voegele, C., Vohle, F.
