Web Survey Bibliography
Relevance & Research Question: Matrix questions are and have always been a big problem in online fieldwork. On the one hand many statistical procedures suggest the use of matrix questions, whilst on the other hand quite a lot of data quality issues are related to matrix questions, - mainly due to satisficing behavior of respondents. One explanation for such a behavior is the high cognitive burden for the respondent when being confronted with the compact presentation of many options and a lot of text on little space. Steps towards a better data quality thus have to find smarter and more intuitive solutions for collecting the needed data. This contribution compares different alternatives for grid questions and shows how to use them in the right context.
Methods & Data: We conducted a study with five splits in total, each of them with a different alternative for the matrix question. Each split contained the respective alternative twice, one time in the context of brand likeability and one time in the context of general attitudes. These concepts have been compared by different metrics in regard to data quality (e.g. variance, time needed on question, consistency with answers in following questions), comparability of the results among these alternatives and the satisfaction of the respondents with usability of each question type. These differences are controlled by sociodemographic factors.
Results: Our results draw a differentiated picture for each of these question types. It seems that there is no universal solution that applies to all contexts. Nonetheless a bunch of recommendations can be drawn for the appropriate design of a questionnaire and the best practice for each of the presented question types.
Added Value: Especially with the growing importance of mobile research in the market research industry, an appropriate questionnaire design gains more and more importance. Smaller screens and new input devices (such as touchscreens) call for alternatives to conventional solutions during fieldwork. We show that researchers most often can count with more satisfied respondents and a better data quality when avoiding the standard matrix question type. Nonetheless they don’t have to do without valuable data.
GOR Homepage (abstract) / (presentation)
Web Survey Bibliography - Web surveys (3865)
- One Drink or Two: Does Quantity Depicted in an Image Affect Web Survey Responses?; 2013; Charoenruk, N., Stange, M.
- A Comparison Between Screen/Follow Item Format and Yes/No Item Format on a Multi-Mode Federal Survey; 2013; Hernandez,S. J., Arakelyan, S. N., Welch, V. E.
- Using Multiple Modes in Follow-Up Contacts in Random-Digit Dialing Surveys; 2013; Chowdhury, P. P.
- Tablets and Smartphones and Netbooks, Oh My! Effects of Device Type on Respondent Behavior; 2013; Ross, H., Mendelson, J., Lackey, M.
- Impacts of Unit Nonresponse in a Recontact Study of Youth; 2013; Mendelson, J., Viera Jr., L.
- Multi-Mode Survey Administration: Does Offering Multiple Modes at Once Depress Response Rates?; 2013; Newsome, J., Levin, K., Langetieg, P., Vigil, M., Sebastiani, M.
- Responsive Design for Web Panel Data Collection; 2013; Bianchi, A., Biffignandi, S.
- Utilizing the Web in a Multi-Mode Survey; 2013; Venkataraman, L.
- Changing to a Mixed-Mode Design: The Role of Mode in Respondents’ Decisions About Participation...; 2013; Collins, D., Mitchell, M., Toomes, M.
- Comparing the Effects of Mode Design on Response Rate, Representativeness, and Cost Per Complete in...; 2013; Tully, R.
- Internet Response for the Decennial Census – 2012 National Census Test; 2013; Reiser, C.
- The Effects of Pushing Web in a Mixed-Mode Establishment Data Collection; 2013; Ellis, C.
- Effects of Lotteries on Response Behavior in Online Panels; 2013; Goeritz, A., Luthe, S. C.
- Lotteries and study results in market research online panels; 2013; Goeritz, A., Luthe, S. C.
- Using Paradata to Study Response to Within-Survey Requests; 2013; Sakshaug, J. W.
- Paradata for Coverage Research ; 2013; Eckman, S.
- Online Fundraising Essentials, Second Edition; 2013; Stevenson, S. C.
- Tips for Evaluating Online Effectiveness; 2013; Stevenson, S. C.
- Ten questions to ask your online survey provider; 2013; Williams, D.
- Survey quality prediction system 2.0; 2013
- Practical tools for designing and weighting survey samples; 2013; Valliant, R. L., Daver, J. A., Kreuter, F.
- Paradata in web surveys; 2013; Callegaro, M.
- A nationwide web-based freight data collection; 2013; Samimi, A., Mohammadian, A., Kawamura, K.
- Mode Matters: Evaluating Response Comparability in a Mixed-Mode Survey; 2013; Bowyer, B. T., Rogowski, J. C.
- Comparing Survey Results Obtained via Mobile Devices and Computers: An Experiment With a Mobile Web...; 2013; de Bruijne, M., Wijnant, A.
- Cognitive Probes in Web Surveys: On the Effect of Different Text Box Size and Probing Exposure on Response...; 2013; Behr, D., Bandilla, W., Kaczmirek, L., Braun, M.
- Best Practice in Online Survey Research with Sensitive Topics; 2013; Kays, K., Keith, T. L., Broughal, M. T.
- Research Intentions are Nothing without Technology: Mixed-Method Web Surveys and the Coberen Wall of...; 2013; Ganassali, S., Rodriguez-Santos, C.
- Reducing Response Burden for Enterprises Combining Methods for Data Collection on the Internet; 2013; Vik, T.
- Measuring Wages Worldwide: Exploring the Potentials and Constraints of Volunteer Web Surveys; 2013; Steinmetz, S., Raess, D., Tijdens, K., de Pedraza, P.
- The Distinctiveness of Online Research: Descriptive Assemblages, Unobtrusiveness, and Novel Kinds of...; 2013; Lanfrey, D.
- Sampling, Channels, and Contact Strategies in Internet Survey; 2013; Macrì, E., Tessitore, C.
- Data Quality in PC and Mobile Web Surveys; 2013; Mavletova, A. M.
- Virtual research assistants: Replacing human interviewers by automated avatars in virtual worlds; 2013; Hasler, B. S., Tuchman, P., Friedman, D.
- Compared to a small, supervised lab experiment, a large, unsupervised web-based experiment on a previously...; 2013; Ryan, R. S., Wilde, M., Crist, S.
- From mixed-mode to multiple devices. Web surveys, smartphone surveys and apps: has the respondent gone...; 2013; Callegaro, M.
- Moving an established survey online – or not?; 2013; Barber, T., Chilvers, D., Kaul, S.
- On the Use of Latent Variable Models to Detect Differences in the Interpretation of Vague Quantifiers...; 2013; Griffin, J.
- An approach to selecting online respondents; 2013; Terhanian, G.
- By the Numbers: Theory of adaptation or survival of the fittest?; 2013; Cavallaro, K.
- Designing and conducting business surveys; 2013; Snijkers, G.,Araldsen, G., , Willimack, D. K.Jones, J.
- Modular Survey Design: A Bite Size Proposal; 2013; Kelly, F., Stevens, S., Johnson, A.
- Cyborgs vs. Monsters: Assembling Modular Surveys to Create Complete Datasets; 2013; Johnson, E. P., Siluk, L., Tarraf, S.
- Do I Have Your Full Attention?; 2013; Cape, P. J.
- Does Sample Size Still Matter?; 2013; Bakken, D. G., Bond, M.
- Optimizing Surveys for Smartphones: Maximizing Response Rates While Minimizing Bias; 2013; Lattery, K., Park Bartolone, G., Saunders, T.
- Shorter Isn't Always Better; 2013; Burdein, I.
- Mobile Research Risk: What Happens to Data Quality When Respondents Use a Mobile Device for a Survey...; 2013; Baker-Prewitt, J.
- Internet-Based Recruitment to a Depression Prevention Intervention: Lessons From the Mood Memos Study...; 2013; Morgan, A. J., Jorm, A. F., Mackinnon, A. J.
- Challenges for Researchers Investigating Contraceptive Use and Pregnancy Intentions of Young Women Living...; 2013; Herbert, D. L., Loxton, D., Bateson, D., Weisberg, E., Lucke, J. C.