Web Survey Bibliography
Title The Low Response Score (LRS): A Metric to Locate, Predict, and Manage Hard-to-Survey Populations
Author Erdman, C.; Bates, N.
Source Public Opinion Quarterly (POQ); 81, 1, pp. 144–156
Year 2016
Database Oxford Journals
Access date 24.08.2017
Abstract In 2012, the US Census Bureau posed a challenge under the
America COMPETES Act, an act designed to improve the competitiveness
of the United States by investing in innovation through research
and development. The Census Bureau contracted Kaggle.com to host
and manage a worldwide competition to develop the best statistical
model to predict 2010 Census mail return rates. The Census Bureau provided
competitors with a block group-level database consisting of housing,
demographic, and socioeconomic variables derived from the 2010
Census, five-year American Community Survey estimates, and 2010
Census operational data. The Census Bureau then challenged teams to
use these data (and other publicly available data) to construct the models.
One goal of the challenge was to leverage winning models as inputs
to a new model-based hard-to-count (HTC) score, a metric to stratify
and target geographic areas according to propensity to self-respond in
sample surveys and censuses. All contest winners employed data-mining
and machine-learning techniques to predict mail-return rates. This
made the models relatively hard to interpret (when compared with the
Census Bureau’s original HTC score) and impossible to directly translate
to a new HTC score. Nonetheless, the winning models contained
insights toward building a new model-based score using variables from
the database. This paper describes the original algorithm-based HTC
score, insights gained from the Census Return Rate Challenge, and the
model underlying a new HTC score.
America COMPETES Act, an act designed to improve the competitiveness
of the United States by investing in innovation through research
and development. The Census Bureau contracted Kaggle.com to host
and manage a worldwide competition to develop the best statistical
model to predict 2010 Census mail return rates. The Census Bureau provided
competitors with a block group-level database consisting of housing,
demographic, and socioeconomic variables derived from the 2010
Census, five-year American Community Survey estimates, and 2010
Census operational data. The Census Bureau then challenged teams to
use these data (and other publicly available data) to construct the models.
One goal of the challenge was to leverage winning models as inputs
to a new model-based hard-to-count (HTC) score, a metric to stratify
and target geographic areas according to propensity to self-respond in
sample surveys and censuses. All contest winners employed data-mining
and machine-learning techniques to predict mail-return rates. This
made the models relatively hard to interpret (when compared with the
Census Bureau’s original HTC score) and impossible to directly translate
to a new HTC score. Nonetheless, the winning models contained
insights toward building a new model-based score using variables from
the database. This paper describes the original algorithm-based HTC
score, insights gained from the Census Return Rate Challenge, and the
model underlying a new HTC score.
Access/Direct link Journal Homepage (abstract) / (full text)
Year of publication2016
Bibliographic typeJournal article
Web survey bibliography - Public Opinion Quarterly (POQ) (90)
- Necessary but Insufficient: Why Measurement Invariance Tests Need Online Probing as a Complementary...; 2017; Meitinger, K.
- Nonresponse in Organizational Surveying: Attitudinal Distribution Form and Conditional Response Probabilities...; 2017; Kulas, J. T.; Robinson, D. H.; Kellar, D. Z.; Smith, J. A.
- Theory and Practice in Nonprobability Surveys: Parallels between Causal Inference and Survey Inference...; 2017; Mercer, A. W.; Kreuter, F.; Keeter, S.; Stuart, E. A.
- Is There a Future for Surveys; 2017; Miller, P. V.
- Respondent mode choice in a smartphone survey ; 2017; Conrad, F. G., Schober, M. F., Antoun, C., Yan, H. Y., Hupp, A., Johnston, M., Ehlen, P., Vickers, L...
- Effects of Mobile versus PC Web on Survey Response Quality: a Crossover Experiment in a Probability...; 2017; Antoun, C.; Couper, M. P.; G. G.Conrad, F. G.
- The Low Response Score (LRS): A Metric to Locate, Predict, and Manage Hard-to-Survey Populations; 2016; Erdman, C.; Bates, N.
- Targeted Appeals for Participation in Letters to Panel Survey Members; 2016; Lynn, P.
- Fieldwork Effort, Response Rate, and the Distribution of Survey Outcomes: A Multilevel Meta-analysis; 2016; Sturgis, P.; Williams, Jo.; Brunton-Smith, I.; Moore, J.
- Measuring Generalized Trust: An Examination of Question Wording and the Number of Scale Points; 2016; Lundmark, S.; Giljam, M.; Dahlberg, S.
- Social Media Analyses for Social Measurement; 2016; Schober, M. F.; Pasek, J.; Guggenheim, L.; Lampe, C.; Conrad, F. G.
- Do Attempts to Improve Respondent Attention Increase Social Desirability Bias?; 2015; Clifford, S.; Jerit, J.
- Response Rates, Nonresponse Bias, and Data Quality: Results from a National Survey of Senior Healthcare...; 2015; Meterko, M.; Restuccia, J. D.; Stolzmann, K.; Mohr, D.; Brennan, C. W.; Glasgow, J.; Kaboli, P.
- Respondent Screening and Revealed Preference Axioms: Testing Quarantining Methods for Enhanced Data...; 2015; Jones, M. S.; House, L. A.; Zhifeng, G.
- Exploring the Effects of Removing "Too Fast" Responses and Respondents from Web Surveys; 2015; Greszki, R.; Meyer, M.; Schoen, H.
- The Effects of the Direction of Rating Scales on Survey Responses in a Telephone Survey; 2015; Keusch, F., Yan, T.
- Assessing the Potential of Paradata and Other Auxiliary Data for Nonresponse Adjustments; 2014; Krueger, B. S., West, B. T.
- Improving Response Rates and Questionnaire Design for Mobile Web Surveys; 2014; de Bruijne, M., Wijnant, A.
- Assessing Within-Household Selection Methods in Household Mail Surveys; 2014; Olson, K., Stange, M., Smyth, J. D.
- Mobile Technologies for Conducting, Augmenting and Potentially Replacing Surveys: Report of the AAPOR...; 2014; Link, M. W., Murphy, J., Schober, M. F., Buskirk, T. D., Childs, J. H., Tesfaye, C.
- Clicking vs. Dragging: Different Uses of the Mouse and Their Implications for Online Surveys; 2014; Sikkel, D., Steenbergen, R., Gras, S.
- The Effect of Benefit Wording on Consent to Link Survey and Administrative Records in a Web Survey; 2014; Sakshaug, J. W., Kreuter, F.
- Video Content in Web Surveys: Effects on Selection Bias and Validity; 2013; Shapiro-Luft, D., Cappella, J.
- Panel Conditioning in Difficult Attitudinal Questions; 2013; Binswanger, J., Schunk, D., Toepoel, V.
- Clarifying Categorical Concepts in a Web Survey.; 2013; Redline, C. D.
- Recruiting A Probability Sample For An Online Panel: Effects Of Contact Mode, Incentives, And Information...; 2012; Scherpenzeel, A., Toepoel, V.
- Does Giving People Their Preferred Survey Mode Actually Increase Survey Participation Rates?; 2012; Olson, K., Smyth, J. D., Wood, H.
- The changing role of address-based sampling in survey research; 2011; Iannacchione, V. G.
- Measuring americans' issue priorities. A new version of the most important problem question reveals...; 2011; Yeager, D. S., Larson, S. B., Krosnick, J. A., Tompson, T.
- Questions for Surveys: Current Trends and Future Directions; 2011; Schaeffer, N. C., Schaeffer, N. C.
- The Future of Modes of Data Collection; 2011; Couper, M. P.
- The Future of Survey Sampling; 2011; Brick, J. M.
- The Impact of “Forgiving” Introductions on the Reporting of Sensitive Behavior in Surveys...; 2011; Peter, J., Valkenburg, P. M.
- Surveying the General Public over the Internet Using Address-Based Sampling and Mail Contact Procedures...; 2011; Messer, B. L., Dillman, D. A.
- Use of Cognitive Shortcuts in Landline and Cell Phone Surveys; 2011; Everett, S. E., Kennedy, C.
- An Alternative to the Response Rate for Measuring a Survey's Realization of the Target Population; 2011; Skalland, B.
- Can Verbal Instructions Counteract Visual Context Effects in Web Surveys?; 2011; Toepoel, V., Couper, M. P.
- Nonresponse Error, Measurement Error, And Mode Of Data Collection: Tradeoffs in a Multi-mode Survey...; 2011; Sakshaug, J. W., Yan, T., Tourangeau, R.
- A Method for Evaluating Mode Effects in Mixed-mode Surveys; 2011; Vannieuwenhuyze, J., Loosveldt, G., Molenberghs, G.
- Total Survey Error: past, present, and future; 2010; Groves, R. M., Lyberg, L. E.
- Research synthesis. AAPOR report on online panels; 2010; Brick, J. M., Baker, R., Blumberg, S. J., Couper, M. P., Courtright, M., Dennis, J. M., Dillman, D....
- Recruiting probability samples for a multi-mode research panel with Internet and mail components; 2010; Rao, K.
- Cell-Phone-Only Voters in the 2008 Exit Poll and Implications for Future Noncoverage Bias ; 2009; Mokrzycki, M., Keeter, S., Kennedy, C.
- Zero Banks: Coverage Error and Bias in Rdd Samples Based on Hundred Banks with Listed Numbers ; 2009; Boyle, J., Bucuvalas, M., Piekarski, L., Weiss, A.
- National Surveys Via RDD Telephone Interviewing vs. the Internet: Comparing Sample Representativeness...; 2009; Chang, L. C., Krosnick, J. A.
- Impact of T-ACASI on Survey Measurements of Subjective Phenomena ; 2009; Harmon, T., Rogers, S. M., Eggleston, E., Roman, A. M., Villarroel, M. A., Chromy, J. R., Ganapathi,...
- Open-Ended Questions in Web Surveys: Can Increasing the Size of Answer Boxes and Providing Extra Verbal...; 2009; Smyth, J. D., Dillman, D. A., Christian, L. M., McBride, M.
- Web Survey Methods: Introduction; 2009; Couper, M. P., Miller, P. V.
- Social desirability bias in CATI, IVR and Web surveys: The effects of mode and question sensitivity; 2008; Kreuter, F., Presser, S., Tourangeau, R.
- Does a Probability-Based Household Panel Benefit from Assignment to Postal Response as an Alternative...; 2008; Rookey, B. D., Hanway, S., Dillman, D. A.