Web Survey Bibliography
Title Evaluating Three Approaches to Statistically Adjust for Mode Effects
Author Kolenikov, S.; Kennedy, C.
Source Journal of Survey Statistics and Methodology, 2, 2, pp. 126-158
Year 2016
Database Oxford Journals
Access date 19.04.2016
Abstract A major hazard in conducting multimode surveys is the potential for mode effects to compromise the response distributions recorded. We evaluate the strengths and weaknesses of three approaches for statistically adjusting for mode effects. Under a regression modeling approach, adjustments are computed by regressing survey responses on mode, demographics, and other relevant variables. Under a multiple imputation approach, mode effects are conceptualized as a missing-data problem. The standard multiple imputation techniques such as chained equations can be used to impute the responses in additional modes. We also propose a new imputation approach based on an econometric framework of implied utilities in logistic regression modeling. We evaluate all three approaches using data from the second wave of the Portraits of American Life Survey sponsored by Rice University's Kinder Institute for Urban Research. This survey featured online and CATI interviewing with a national sample of adults, with random assignment to either CATI-only or web with CATI follow-up for nonrespondents. We develop a workflow to determine which variables require a mode effects adjustment based on standard false discovery rate multiple hypothesis testing procedures. We detected a significant mode effect on four survey outcomes after controlling for demographics and risk of type I error. The mode effects adjustments were then applied to these variables. The effects on the standard errors and point estimates are examined and discussed along with the advantages and disadvantages of each adjustment approach. The multiple imputation approach produced estimates with better apparent accuracy, as evidenced by better internal consistency of the estimates and a moderate increase in the standard errors. Unlike the regression adjustment approach, which can only produce aggregated estimates for the whole study, the multiple imputation approach can be used for disaggregated analysis with mode-adjusted estimates as well.
Access/Direct link Oxford Journals (Abstract) / (Full text)
Year of publication2014
Bibliographic typeJournal article
Web survey bibliography - 2014 (234)
- Detecting Insufficient Effort Responding with an Infrequency Scale: Evaluating Validity and Participant...; 2016; Huang, J. L.; Bowling, N. A.; Liu, Me.; Li, Yu.
- Evaluating Three Approaches to Statistically Adjust for Mode Effects; 2016; Kolenikov, S.; Kennedy, C.
- An Examination of Opposing Responses on Duplicated Multi-Mode Survey Responses; 2016; Djangali, A.
- Computer-assisted and online data collection in general population surveys; 2016; Skarupova, K.
- Usability of the ACS Internet Instrument on Mobile Devices; 2015; Horwitz, R.
- Explorations in Non - Probability Sampling Using the Web; 2015; Brick, J. M.
- On Bias Adjustments for Web Surveys; 2015; Fan, L.; Lou, W.; Landsman, V.
- Are they willing to use the web? First results of a possible switch from PAPI to CAPI/CAWI in an establishment...; 2015; Ellguth, P.; Kohaut, S.
- Web panel surveys – a challenge for official statistics; 2015; Svensson, J.
- Estimation with Non-probability Surveys and the Question of External Validity; 2015; Dever, J. A.; Valliant, R. L.
- Measurement Properties of Web Surveys; 2015; Tourangeau, R.
- Improving Response to Household Surveys Using Mail Contact to Request Responses over the Internet: Results...; 2015; Dillman, D. A.
- The quality of data collected using online panels: a decade of research ; 2015; Callegaro, M.
- Sub-optimal Respondent Behavior and Data Quality in Online Surveys; 2015; Thomas, R. K.
- Methodology of the RAND Mid-Term 2014 Election Panel; 2015; Carman, K. G; Pollack, S.
- Designing Bonsai Surveys: The small but perfectly formed survey experience to meet the needs of the...; 2015; Puleston, J.
- Suggestions for international research using electronic surveys; 2015; e Silva, S. C.; Duarte, P.
- Recruiting Respondents for a Mobile Phone Panel: The Impact of Recruitment Question Wording on Cooperation...; 2015; Busse, B.; Fuchs, M.
- The effect of multiple reminders on response patterns in a Danish health survey; 2015; Christensen, A. I.; Ekholm, O.; Kristensen, P. L.; Larsen, F. B.; Vinding, A. L.; Gluemer, C.; Juel,...
- The quality of responses to grid questions as used in Web questionnaires (compared with paper questionnaires...; 2015; Dominguez, J. A.; de Rada, V. D.
- Identifying predictors of survey mode preference; 2015; Millar, M. M.; Olson, K.; Smyth, J. D.
- The Impact of Mixing Modes on Reliability in Longitudinal Studies; 2014; Cernat, A.
- Growing Beyond the Phone Tree; 2014; Hayzlett, J.
- A Comparison of Different Online Sampling Approaches for Generating National Samples; 2014; Heen, M. S. J., Lieberman, J. D., Miethe, T. D.
- Does Sequence Matter in Multimode Surveys: Results from an Experiment; 2014; Wagner, J., Arrieta, J., Guyer, H., Ofstedal, M. B.
- The Use of Cognitive Interviewing Methods to Evaluate Mode Effects in Survey Questions; 2014; Gray, M., Blake, M., Campanelli, P.
- A Mixed Methods Approach to Network Data Collection; 2014; Rice, E., Holloway, I. W., Barman-Adhikari, A., Fuentes, D., Brown, C. H., Palinkas, L. A.
- Infliential Factors on Survey Outcomes: Length of Survey, Device Selection and Extrnal Elements; 2014; Ribeiro, E.
- The Effect of Mobile Web Survey Design on Screen Orientation Manipulation; 2014; Young, R.H.; Crawford, S. D.; Couper, M. P.; Nelson, T. F.
- Investigating Response Quality in Mobile and Desktop Surveys: A Comparison of Radio Buttons, Visual...; 2014; Toepoel, V.; Funke, F.
- Do online access panels really need to allow and adapt surveys to mobile devices? ; 2014; Revilla, M.; Toninelli, D.; Ochoa, C.; Loewe, G.
- Why you need to make your surveys mobile friendly NOW; 2014; Lorch, J.; Mitchell, N.
- Assessing the Impact Device Choice Has on Web Survey Data Collection ; 2014; Hupp, A.; Schroeder, H. M.; Piskorowski, A.D.
- Understanding Mobility: Consent and Capture of Geolocation Data in Web Surveys; 2014; Crawford, S. D.; McClain, C.; Young, R.H.; Nelson, T. F.
- Swipe, Snap & Chat: Mobile Survey Data Collection Using Touch Question Types and Mobile OS Features ; 2014; Buskirk, T. D.; Michaud, J.; Saunders, T.
- Statistical Approaches to Analyze Self-Reported Susceptibility to Driver Distraction; 2014; Chen, H-Y. W.; Donmez, B.; Ko, Y-D.
- Using Web Panels for Official Statistics; 2014; Bethlehem, J.
- The problem of non-response in population surveys on the topic of HIV and sexuality: a comparative study...; 2014; Wallander, L.; H.; Mannheimer, L. N.; Oestergren, P. O.; Plantin, L.Tikkanen, R. H.
- Does the Length of Fielding Period Matter? Examining Response Scores of Early Versus Late Responders; 2014; Dyer Yount, N.; Lewis, T.; Lee, K.; Sigman, R.
- FocusVision 2014 Annual MR Technology Report; 2014; Macer, T., Wilson, S.
- When it comes to mobile respondent experience and data quality, survey design matters; 2014; Mitchell, N.
- The Changing Landscape of Technology and its Effect on Online Survey Data Collection; 2014; Mitchell, N.
- Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method, 4th Edition; 2014; Dillman, D. A., Smyth, J. D., Christian, L. M.
- The survey playbook: how to create the perfect survey. (Vol.1); 2014; Champagne, M. V.
- Do your own online surveys. DYI and self serve market research; 2014; Cary, N.
- The Influence of Answer Box Format on Response Behavior on List-Style Open-Ended Questions; 2014; Keusch, F.
- Nonprobability Web Surveys to Measure Sexual Behaviors and Attitudes in the General Population: A Comparison...; 2014; Erens, B.; Burkill, S.; Couper, M. P.; C., Clifton, S., Tanton, C., Phelps, A., Datta, J., Mercer,...
- Luteal-phase support in assisted reproduction treatment: real-life practices reported worldwide by an...; 2014; Vaisbuch, E., de Ziegler, D., Leong, M., Shoham, Z., Weissman, A.
- Facebook, Twitter, & Qr Codes: An Exploratory Trial Examining The Feasibility Of Social Media Mechanisms...; 2014; Gu, L. L.
- Time-dependent variation in the responses to the web-based ISAAC questionnaire; 2014; Yoshida, K., Sasaki, M., Odajima, H., Itazawa, T., Hashimoto, K., Furukawa, M., Adachi, Y.