Web Survey Bibliography
Relevance & Research Question
The market research industry is wedded to quota controls. We apply Age and Gender quotas without a second thought as to why or indeed whether they are doing any good at all. Our argument is that, in the modern online sampling world, a different set of stratifications must be applied and our old assumptions simply do not apply. Why not? The answer, in common with so many of the problems in sampling in online research, lies in the frame. The frame in traditional research was close to the population; therefore a quota controlled random sample would tend to produce samples that, within the quota strata, also contained representative numbers of all other attitudes and behaviours. This is not the case with online access panels.
Methods and Data
Our experiment uses our US panel; the topic, eye colour, is unrelated to Age and Gender but is strongly related to Ethnicity. We have conducted 2 samples. The first strictly controlled on Age, Gender and Region, the second controlled on Ethnicity alone. Our Age Gender Region ‘nat rep’ sample should underestimate the number with brown eyes. The Ethnicity we expect to estimate eye colour extremely well. At the same time a third sample will be drawn which is simply “random enough”. Our expectation is that this sample will also under-perform on eye colour but will equal the findings from “nat rep” sample 1. A second experiment will be undertaken where the variable of interest is unrelated to anything –left- or right-handedness. Our hypothesis is that all three samples will perform equally well.
The results are precisely as predicted.
Researchers, particularly in the commercial world, apply quota controls to ensure “representivity” as a matter of practice, they do it because they have been told to, it is part of the folklore of market research. This is not sustainable in a world where we are no longer dealing with essentially incomplete frames. More science and less folklore needs to be applied to make the best of an increasing unscientific world.
Conference Homepage (abstract) / (presentation)
Web Survey Bibliography (6389)
- Measure the response burden in the Swedish Intrastat system; 2012; Weideskog, F.
- Mode and non-response effects and their treatment; 2012; Chrysanthopoulos, S., Georgostathi, A.
- What can be said about quality in the Central Population Register based on a self-completion survey...; 2012; Falnes-Dalheim, E., Pedersen, H. E.
- Improving the quality of complex surveys: The case of the EU Labour Force Survey ; 2012; van der Valk, J.
- Pros and cons of Internet based User Satisfaction Surveys; 2012; Consoli, A., Matsulevits, L.
- The re-engineering of the Structural Earnings survey process: Mixed - Mode data collection and new E...; 2012; Cardinaleschi, S., De Santis, S., Rocci, F., Spinelli, V.
- Between demand and reality: Ensuring efficiency and quality in pretesting questionnaires; 2012; Sattelberger, S., Blanke, K.
- How to provide high data quality in online-questionnaires: Setting guidelines in design; 2012; Tries, S., Nebel, S., Blanke, K.
- Boosting Web pick-up Rates by referring to Compliance Principles ; 2012; Falnes-Dalheim, E., Haraldsen, G., Sundvoll, A.
- Choosing a Data Collection Approach: Mixed Mode Design Experiences in Statistics Finland; 2012; Taskinen, P., Kiianmaa, N.
- Ebook readings jumps, print book reading declines; 2012; Rainie, L., Duggan, M.
- Does mentioning "Some People" and "Other People" in an opinion question improve...; 2012; Yeager, D. S., Krosnick, J. A.
- Digital Divides: A connectivity continuum for the United States. Data from the 2011 Current Population...; 2012; File, T.
- Developments and the impact of smart technology; 2012; Macer, T.
- How Should Debriefing Be Undertaken in Web-Based Studies? Findings From a Randomized Controlled Trial...; 2012; McCambridge, J., Kypri, K., Wilson, A.
- Better customer in sight in real time; 2012; Macdonald, E., Wilson, H. N., Konus, H.
- Best practices in data cleaning: A complete guide to everything you need to do before and after collecting...; 2012; Osborne, J. W.
- Benchmarking for better surveys; 2012; Nallan, S.
- An experimental investigation of the effects of noncontingent and contingent incentives in recruiting...; 2012; Lavrakas, P. J., Dennis, J. M., Peugh, J., Shand-Lubbers, J., Lee, E., Peugh, J., Charlebois, O., Murakami...
- Adult gadget ownership over time (2006-2012); 2012
- Subjective Well-being Of Spanish Workers: Continuous Voluntary Web Survey Examination; 2012; de Pedraza, P., Guzi, M.
- 28 Questions to Help Buyers of Online Samples; 2012
- 2010 ACS Content Test Evaluation Report Covering Computer and Internet ; 2012; Shin, H. B.
- Specific mixed-mode methodology to reach sensory disabled people in quantitative surveys; 2012; Fontaine, S.
- Response Mode Choice and the Hard-to-Interview in the American Community Survey; 2012; Nichols, E. M., Horwitz, R., Guarino Tancreto, J.
- Recruiting in an Internet panel using respondent driven sampling; 2012; Schonlau, M.
- A Choice in Mode: A Solution for Increasing Response Rates of Hard-to-Survey Populations?; 2012; Haan, M., Ongena, Y. P.
- The Feasibility of Conducting a Web Survey Using Respondent Driven Sampling among Transgenders in the...; 2012; Kappelhof, J.
- The role of topic interest and topic salience in online panel web surveys.; 2012; Keusch, F.
- Multi-Language Multi-Continent B2B Community Panel: How B2B research can effectively span the world; 2012; Morden, M., Accomando, E.
- Can Survey Gaming Techniques Cross Continents? Examining cross cultural reactions to creative questioning...; 2012; Puleston, J.
- Facing The Future Webcams as a survey tool in China; 2012; Gordon, A., Llewellyn, T., Gu, E.
- Device Diversity: Understanding the complexity of varied devices for taking surveys – Case study...; 2012; Pearson, C., Backlund, K., Veling, L., Tsvelik, M., Jehoel, S.
- Research Goes Mobile: Findings from initial smartphone application research; 2012; Dubreuil, C., Joubert, S.
- Research in the Mobile Mindset: Exploring the unexplored in the mobile research space; 2012; Willems, A., Veris, E., Verhaeghe, A.
- Better Answers to Basic Questions: Enhancing the accuracy of online reach and audience metrics; 2012; van Dam, P. H., van Ossenbruggen, R., Voorend, R.
- Rules of engagement: The war against poorly engaged respondents - guidelines for elimination; 2012; Gittelman, S. H., Trimarchi, E.
- Reality check in the digital age: The relationship between what we ask and what people actually do; 2012; Hofmeyr, J., Louw, A.
- Website Versus Traditional Survey Comments: Do they tell the same story?; 2012; Brandt, R., House, M.
- Dimensions of Online Survey Data Quality What really matters?; 2012; Puleston, J., Eggers, M.
- WEBDATANET: web-based data-collection methodological challenges, solutions and implementations. Action...; 2012; de Pedraza, P.
- WebSM Study: Survey software features overview ; 2012; Vehovar, V.; Cehovin, G.; Kavcic, L.; Lenar, J.
- Web Panels; 2012; Bethlehem, J., Biffignandi, S.
- Use of Response Propensities; 2012; Bethlehem, J., Biffignandi, S.
- Weighting Adjustment Techniques; 2012; Bethlehem, J., Biffignandi, S.
- The Problem of Self-Selection; 2012; Bethlehem, J.,Biffignandi, S.
- The Problem of Undercoverage; 2012; Bethlehem, J., Biffignandi, S.
- Mixed-Mode Surveys; 2012; Bethlehem, J., Biffignandi, S.
- Designing a Web Survey Questionnaire; 2012; Bethlehem, J., Biffignandi, S.
- Examining Contexts-of-Use for Web-Based and Paper-Based Questionnaires; 2012; Hardré, P. L., Crowson, H. M., Xie, K.