Web Survey Bibliography
Relevance & Research Question
The market research industry is wedded to quota controls. We apply Age and Gender quotas without a second thought as to why or indeed whether they are doing any good at all. Our argument is that, in the modern online sampling world, a different set of stratifications must be applied and our old assumptions simply do not apply. Why not? The answer, in common with so many of the problems in sampling in online research, lies in the frame. The frame in traditional research was close to the population; therefore a quota controlled random sample would tend to produce samples that, within the quota strata, also contained representative numbers of all other attitudes and behaviours. This is not the case with online access panels.
Methods and Data
Our experiment uses our US panel; the topic, eye colour, is unrelated to Age and Gender but is strongly related to Ethnicity. We have conducted 2 samples. The first strictly controlled on Age, Gender and Region, the second controlled on Ethnicity alone. Our Age Gender Region ‘nat rep’ sample should underestimate the number with brown eyes. The Ethnicity we expect to estimate eye colour extremely well. At the same time a third sample will be drawn which is simply “random enough”. Our expectation is that this sample will also under-perform on eye colour but will equal the findings from “nat rep” sample 1. A second experiment will be undertaken where the variable of interest is unrelated to anything –left- or right-handedness. Our hypothesis is that all three samples will perform equally well.
The results are precisely as predicted.
Researchers, particularly in the commercial world, apply quota controls to ensure “representivity” as a matter of practice, they do it because they have been told to, it is part of the folklore of market research. This is not sustainable in a world where we are no longer dealing with essentially incomplete frames. More science and less folklore needs to be applied to make the best of an increasing unscientific world.
Conference Homepage (abstract) / (presentation)
Web Survey Bibliography - 2011 (584)
- Involve while you evolve. How to make mobile research work for everyone; 2011; Luck, K.
- Internet access quarterly update 2011 Q1; 2011
- In search of a new approach to measure newspaper audiences in Canada: The journey continues; 2011; Crassweller, A., Rogers, J., Graves, F., Gauthier, E., Charlebois, O.
- Households with Computers, Telephone Subscriptions, and Internet Access, Selected Years, 1997 - 2010; 2011
- Harvard research data security policy; 2011
- Handbook of statistical data editing and imputation; 2011; Waal, T., Pannekoek, J.; Scholtus, S.
- Handbook of parametric and nonparametric statistical procedures, 5th Edition; 2011; Sheskin, D. J.
- GRE® program announces big benefits and big savings for GRE® test takers worldwide; 2011
- Google and Kantar develop measurement panel; 2011
- Going online with assessment; 2011; Burke, E. et al.
- Global market research 2011; 2011
- Focus Group Methodology: Principle and Practice; 2011; Liamputtong, P.
- External preference mapping of commercial antiaging creams based on consumers' responses to a check...; 2011; Parente, M. E., Manzoni, A. V., Ares, G.
- Exploring the digital nation. Computer and Internet use at home; 2011
- Experiments for evaluating survey questions; 2011; Krosnick, J. A.
- Evaluating the usability of personal digital assistants to collect behavioral data on adolescents with...; 2011; McClamroch, K. J
- Eurobarometer Special surveys: EB75.1 E-Communications Household Survey. Special Eurobarometer 362; 2011
- Survey Data Quality Provisions in Statistics Canada E-Questionnaire Solution: Retrospective and Perspectives...; 2011; Abiza, Y.
- Developing Electronic Questionnaire Guidelines: Issues and Challenges in a Changing Environment; 2011; Cote, A.-M., Kelly, P., Lawrence, D.
- Triton: a general tool for data collection and micro editing; 2011; Erikson, J.
- A Generalized System for Aided Development and Monitoring of Web Surveys; 2011; Torelli, R.
- Ethical issues in Internet research; 2011; Hoerger, M., Currell, C.
- Using survey data collection as a tool for improving the survey process; 2011; Biffignandi, S., Perani, G., Laureti, A.
- Essential methods for design based sample surveys; 2011; Pfeffermann, D., Rao, C. R.
- ESOMAR AND CASRO submission to the W3C tracking protection working group - Market research techniques...; 2011
- A picnic in the field; negotiating the presentation of the self in researcher/respondent relationships...; 2011; Parsons, J.
- The benefits and constraints of e-mail interviews and discussions as methods of accessing valid data; 2011; Roberts, A.
- Is There a Quick Fix for Open-ended Questions? A Comparison of Qualitative Analysis Techniques; 2011; Tesfaye, C.
- The Impact of Open-Ended Questions: A Multivariate Study of Respondent Engagement; 2011; Gittelman, S. H.
- “You are Invited to Participate”: Challenges of Applying Mixed Survey Methods to Assess...; 2011; Chew, F.
- Literacy and Data Quality in Self-Administered Surveys; 2011; Smyth, J. D., Olson, K.
- Catch Them When You Can: Speeders and Their Role in Online Data Quality; 2011; Gutierrez, C., Wells, T., Rao, K., Kurzynski, D.
- Effects of Post-Incentives on Response Rates, Costs, and Response Quality in a Web Survey of College...; 2011; Stevenson, J., Dykema, J., Cyffka, K., Klein, L., Goldrick-Rab, S.
- Observed Differences in the Placement and Wording of Neutral Response Options in Web Surveys: An Experiment...; 2011; Walton, L., Cobb, C. L., DiSogra, C.
- Effects of response format on requalification for recontact studies; 2011; Thomas, R. K.
- A meta-analysis of experiments manipulating progress indicators in Web surveys; 2011; Callegaro, M., Villar, A., Yang, Y.
- Does mentioning "some people" and "other people" in a survey question increase the...; 2011; Yeager, D. S., Krosnick, J. A.
- Do not track gathers momentum; 2011; Stark, D.
- “Don’t know” the difference - An experimental comparison between Web and CATI; 2011; Schielicke, A.-M., Degen, M.
- Display resolution; 2011
- A Survey Stopping Rule Based on Weighting for Unit Nonresponse; 2011; Lewis, T.
- Classic Inspirations for Social Research Methodology in the time of Online Access Panels ; 2011; Jeřábek, H.
- Five Things You Should Know About Mobile Data Collection; 2011; Pingitore, G.
- Mixed Methods - Analyzing Open-Ended Comments in a Quantitative Employee Survey; 2011; Lawton, L., Broege, N.
- Changing Survey Methods (Discussion); 2011; Lavrakas, P. J.
- Code of standards and ethics for survey research; 2011
- Causes of survey incompletes: Why panelists say they abandon surveys; 2011; Henning, J.
- Canadian online panels: Similar or different?; 2011; Chan, P., Ambrose, D.
- Blend, balance, and stabilize respondent sources; 2011; Eggers, M., Drake, E.
- Beyond data stability: Rising above quality concerns; 2011