Web Survey Bibliography
Title Evaluating Three Approaches to Statistically Adjust for Mode Effects
Author Kolenikov, S.; Kennedy, C.
Source Journal of Survey Statistics and Methodology, 2, 2, pp. 126-158
Year 2016
Database Oxford Journals
Access date 19.04.2016
Abstract A major hazard in conducting multimode surveys is the potential for mode effects to compromise the response distributions recorded. We evaluate the strengths and weaknesses of three approaches for statistically adjusting for mode effects. Under a regression modeling approach, adjustments are computed by regressing survey responses on mode, demographics, and other relevant variables. Under a multiple imputation approach, mode effects are conceptualized as a missing-data problem. The standard multiple imputation techniques such as chained equations can be used to impute the responses in additional modes. We also propose a new imputation approach based on an econometric framework of implied utilities in logistic regression modeling. We evaluate all three approaches using data from the second wave of the Portraits of American Life Survey sponsored by Rice University's Kinder Institute for Urban Research. This survey featured online and CATI interviewing with a national sample of adults, with random assignment to either CATI-only or web with CATI follow-up for nonrespondents. We develop a workflow to determine which variables require a mode effects adjustment based on standard false discovery rate multiple hypothesis testing procedures. We detected a significant mode effect on four survey outcomes after controlling for demographics and risk of type I error. The mode effects adjustments were then applied to these variables. The effects on the standard errors and point estimates are examined and discussed along with the advantages and disadvantages of each adjustment approach. The multiple imputation approach produced estimates with better apparent accuracy, as evidenced by better internal consistency of the estimates and a moderate increase in the standard errors. Unlike the regression adjustment approach, which can only produce aggregated estimates for the whole study, the multiple imputation approach can be used for disaggregated analysis with mode-adjusted estimates as well.
Access/Direct link Oxford Journals (Abstract) / (Full text)
Year of publication2014
Bibliographic typeJournal article
Web survey bibliography - Kennedy, C. (11)
- Evaluating Three Approaches to Statistically Adjust for Mode Effects; 2016; Kolenikov, S.; Kennedy, C.
- Cell RDD Respondents Unmasked: Progress Report on Geo and Demo Appends to the Wireless Frame; 2015; DiSogra, C.; Kennedy, C.Mosher, M.
- Mode effect analysis and adjustment in a split-sample mixed-mode Web/CATI survey; 2013; Kolenikov, S., Kennedy, C.
- Use of Cognitive Shortcuts in Landline and Cell Phone Surveys; 2011; Everett, S. E., Kennedy, C.
- Nonresponse and Measurement Error in Mobile Phone Surveys ; 2010; Kennedy, C.
- Measurement Error in Cell Phone Surveys; 2009; Kennedy, C., Everett, S. E., Traugott, M. W.
- Mechanisms of Nonresponse in Cell Phone Surveys; 2009; Kennedy, C., Everett, S. E., Traugott, M. W.
- Interactive Interventions in Web Surveys Can Increase Respondent Conscientiousness; 2009; Conrad, F. G., Tourangeau, R., Couper, M. P., Kennedy, C.
- Cell-Phone-Only Voters in the 2008 Exit Poll and Implications for Future Noncoverage Bias ; 2009; Mokrzycki, M., Keeter, S., Kennedy, C.
- Costs and Benefits from Conducting a Supplemental Sample of Cell Phone Numbers in National RDD Surveys...; 2008; Kennedy, C.
- The Impact of the Spacing of the Scale Options in a Web Survey; 2008; Kennedy, C., Tourangeau, R., Conrad, F. G., Couper, M. P., Redline, C. D.