Web Survey Bibliography
Title Evaluating Three Approaches to Statistically Adjust for Mode Effects
Author Kolenikov, S.; Kennedy, C.
Source Journal of Survey Statistics and Methodology, 2, 2, pp. 126-158
Year 2016
Database Oxford Journals
Access date 19.04.2016
Abstract A major hazard in conducting multimode surveys is the potential for mode effects to compromise the response distributions recorded. We evaluate the strengths and weaknesses of three approaches for statistically adjusting for mode effects. Under a regression modeling approach, adjustments are computed by regressing survey responses on mode, demographics, and other relevant variables. Under a multiple imputation approach, mode effects are conceptualized as a missing-data problem. The standard multiple imputation techniques such as chained equations can be used to impute the responses in additional modes. We also propose a new imputation approach based on an econometric framework of implied utilities in logistic regression modeling. We evaluate all three approaches using data from the second wave of the Portraits of American Life Survey sponsored by Rice University's Kinder Institute for Urban Research. This survey featured online and CATI interviewing with a national sample of adults, with random assignment to either CATI-only or web with CATI follow-up for nonrespondents. We develop a workflow to determine which variables require a mode effects adjustment based on standard false discovery rate multiple hypothesis testing procedures. We detected a significant mode effect on four survey outcomes after controlling for demographics and risk of type I error. The mode effects adjustments were then applied to these variables. The effects on the standard errors and point estimates are examined and discussed along with the advantages and disadvantages of each adjustment approach. The multiple imputation approach produced estimates with better apparent accuracy, as evidenced by better internal consistency of the estimates and a moderate increase in the standard errors. Unlike the regression adjustment approach, which can only produce aggregated estimates for the whole study, the multiple imputation approach can be used for disaggregated analysis with mode-adjusted estimates as well.
Access/Direct link Oxford Journals (Abstract) / (Full text)
Year of publication2014
Bibliographic typeJournal article
Web survey bibliography (4086)
- Web Surveys Versus Other Survey Modes: An Updated Meta-analysis Comparing Response Rates ; 2016; Wengrzik, J.; Bosnjak, M.; Lozar Manfreda, K.
- The Effect of a Pre-due Date Reminder Letter on Non response in a Business Survey ; 2016; Hernandez, A. D.; Fan, C. C.; Tuttle, A.
- Adapting the Alternative Questionnaire Experiment for a Telephone Survey: Preparing for Changes to the...; 2016; Patten, E.; Brown, A.; Parker, K.
- Retrospective Measurement of Students’ Extracurricular Activities with a Self-administered Calendar...; 2016; Furthmueller, P.
- Privacy Concerns in Responses to Sensitive Questions. A Survey Experiment on the Influence of Numeric...; 2016; Bader, F., Bauer, J., Kroher, M., Riordan, P.
- Ballpoint Pens as Incentives with Mail Questionnaires – Results of a Survey Experiment; 2016; Heise, M.
- Non-Observation Bias in an Address-Register-Based CATI/CAPI Mixed Mode Survey; 2016; Lipps, O.
- Spatial Modeling through GIS to Reveal Error Potent ial in Survey Data: Where, What and How Much ; 2016; English, N.; Ventura, I.; Bilgen, I.; Stern, M. J.
- Bees to Honey or Flies to Manure? How the Usual Subject Recruitment Exacerbates the Shortcomings of...; 2016; Snell, S. A., Hillygus, D. S.
- Thinking Inside the Box Visual Design of the Response Box Affects Creative Divergent Thinking in an...; 2016; Mohr, A. H.; Sell, A.; Lindsay, T.
- Detecting Insufficient Effort Responding with an Infrequency Scale: Evaluating Validity and Participant...; 2016; Huang, J. L.; Bowling, N. A.; Liu, Me.; Li, Yu.
- Detecting careless respondents in web-based questionnaires: Which method to use?; 2016; Niesen, A. S. M.; Meijer, R. R.; Tendeiro, J. N.
- Web surveys for offline rural communities ; 2016; Gichohi, B. W.
- On-line life history calendar and sensitive topics: A pilot study; 2016; Morselli, D.; Berchtold, A.; Granell, J.-C. S.; Berchtold, And.
- Does survey mode matter for studying electoral behaviour? Evidence from the 2009 German Longitudinal...; 2016; Bytzek, E.; Bieber, I. E.
- The impact of visual design and response formats on data quality in a web survey of MOOC students; 2016; Maloshonok, N.; Terentev, E.
- An experiment comparing grids and item-by-item formats in web surveys completed through PCs and smartphones...; 2016; Revilla, M.; Toninelli, D.; Ochoa, C.
- Establishing the accuracy of online panels for survey research; 2016; Bruggen, E.; van den Brakel, J.; Krosnick, J. A.
- Gamifying Questions Using Text Alone; 2016; Cape, P. J.
- Assessing the Effects of Participant Preference and Demographics in the Usage of Web-based Survey Questionnaires...; 2016; Mlikotic, R.; Parker, B.; Rajapakshe, R.
- Improving Inpatient Surveys: Web-Based Computer Adaptive Testing Accessed via Mobile Phone QR Codes; 2016; Chien, T. S.; Lin, W.S.
- Surveying End-of-Life Medical Decisions in France: Evaluation of an Innovative Mixed-Mode Data Collection...; 2016; Legleye, S; Pennec, S.; Monnier, A.; Stephan, A.; Brouard, N.; Bilsen, J.; Cohen, J.
- Problems and Prospects in Survey Research; 2016; Moy, P.; Murphy, J.
- When will Nonprobability Surveys Mirror Probability Surveys? Considering Types of Inference and Weighting...; 2016; Pasek, J.
- Eye-tracking Social Desirability Bias; 2016; Kaminska, O.; Foulsham, T.
- Evaluating Three Approaches to Statistically Adjust for Mode Effects; 2016; Kolenikov, S.; Kennedy, C.
- Distractions: The Incidence and Consequences of Interruptions for Survey Respondents ; 2016; Ansolabehere, S.; Schaffner, B. F.
- The Effect of CATI Questions, Respondents, and Interviewers on Response Time; 2016; Olson, K.; Smyth, J. D.
- Pre-Survey Text Messages (SMS) Improve Participation Rate in an Australian Mobile Telephone Survey:...; 2016; Dal Grande, E.; Chittleborough, C. R.; Campostrini, S.; Dollard, M.; Taylor, A. W.
- Pitfalls, Potentials, and Ethics of Online Survey Research: LGBTQ and Other Marginalized and Hard-to...; 2016; McInroy, L. B.
- Effects of Personalization and Invitation Email Length on Web-Based Survey Response Rates; 2016; Trespalacios, J. H.; Perkins, R. A.
- Linearization Variance Estimators for Mixed ‒ mode Survey Data when Response Indicators are Modeled...; 2016; Demnati, A.
- Forecasting proportional representation elections from non-representative expectation surveys; 2016; Graefe, A.
- Short and Sweet? Length and Informative Content of Open-Ended Responses Using SMS as a Research Mode; 2016; Walsh, E.; Brinker, J. K.
- Adaptive survey designs to minimize survey mode effects – a case study on the Dutch Labor Force...; 2016; Calinescu, M.; Schouten, B.
- Mixing modes of data collection in Swiss social surveys: Methodological report of the LIVES-FORS mixed...; 2016; Roberts, C.; Joye, D.; Staehli, M. E.
- What is the gain in a probability-based online panel to provide Internet access to sampling units that...; 2016; Revilla, M.; Cornilleau, A.; Cousteaux, A-S.; Legleye, S; de Pedraza, P.
- Representative web-survey!; 2016; Linde, P.
- Assessing targeted approach letters: effects in different modes on response rates, response speed and...; 2016; Lynn, P.
- New Generation of Online Questionnaires?; 2016; Revilla, M.; Ochoa, C.; Turbina, A.
- The Analysis of Respondent’s Behavior toward Edit Messages in a Web Survey; 2016; Park, Y.
- Refining the Web Response Option in the Multiple Mode Collection of the American Community Survey; 2016; Hughes, T.; Tancreto, J.
- The Utility of an Online Convenience Panel for Reaching Rare and Dispersed Populations; 2016; Sell, R.; Goldberg, S.; Conron, K.
- Assessment of Innovations in Data Collection Technology for Understanding Society; 2016; Couper, M. P.
- Comparing online and telephone survey results in the context of a skin cancer prevention campaign evaluation...; 2016; Hollier, L.P.; Pettigrew, S.; Slevin, T.; Strickland, M.; Minto, C.
- Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk; 2016; Berinsky, A.; Huber, G. A.; Lenz, G. S.
- Setting Up an Online Panel Representative of the General Population The German Internet Panel; 2016; Blom, A. G.; Gathmann, C.; Krieger, U.
- Implementation of Web-Based Respondent Driven Sampling among Men Who Have Sex with Men in Sweden; 2016; Stroemdahl, S.; Lu, X.; Bengtsson, L.; Liljeros, F.; Thorson, A.
- Options for Fielding and Analyzing Web Surveys; 2016; Schonlau, M.; Couper, M. P.
- Report of the Inquiry into the 2015 British general election opinion polls; 2016; Sturgis, P., Baker, N., Callegaro, M., Fisher, St., Green, J., Jennings, W., Kuha, J., Lauderdale, B...